2023-02-13 16:53:33,546 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-13 16:53:33,845 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd 2023-02-13 16:53:33,861 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 16:53:33,862 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc, deleteOnExit=true 2023-02-13 16:53:33,862 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 16:53:33,862 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/test.cache.data in system properties and HBase conf 2023-02-13 16:53:33,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 16:53:33,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/hadoop.log.dir in system properties and HBase conf 2023-02-13 16:53:33,864 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 16:53:33,864 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 16:53:33,864 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 16:53:33,984 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 16:53:34,401 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 16:53:34,406 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 16:53:34,407 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 16:53:34,407 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 16:53:34,408 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 16:53:34,408 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 16:53:34,409 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 16:53:34,409 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 16:53:34,410 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 16:53:34,410 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 16:53:34,411 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/nfs.dump.dir in system properties and HBase conf 2023-02-13 16:53:34,411 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/java.io.tmpdir in system properties and HBase conf 2023-02-13 16:53:34,411 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 16:53:34,412 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 16:53:34,412 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 16:53:34,987 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 16:53:34,991 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 16:53:35,308 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 16:53:35,486 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 16:53:35,505 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 16:53:35,546 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 16:53:35,579 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/java.io.tmpdir/Jetty_localhost_42315_hdfs____k1v75v/webapp 2023-02-13 16:53:35,722 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42315 2023-02-13 16:53:35,759 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 16:53:35,759 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 16:53:36,228 WARN [Listener at localhost/45789] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 16:53:36,354 WARN [Listener at localhost/45789] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 16:53:36,382 WARN [Listener at localhost/45789] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 16:53:36,392 INFO [Listener at localhost/45789] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 16:53:36,400 INFO [Listener at localhost/45789] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/java.io.tmpdir/Jetty_localhost_34199_datanode____5248vr/webapp 2023-02-13 16:53:36,500 INFO [Listener at localhost/45789] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34199 2023-02-13 16:53:36,883 WARN [Listener at localhost/36833] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 16:53:37,440 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcc26de3b869903fd: Processing first storage report for DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b from datanode 71870c0e-015e-4301-a526-b4af394d64df 2023-02-13 16:53:37,442 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcc26de3b869903fd: from storage DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b node DatanodeRegistration(127.0.0.1:44175, datanodeUuid=71870c0e-015e-4301-a526-b4af394d64df, infoPort=43515, infoSecurePort=0, ipcPort=36833, storageInfo=lv=-57;cid=testClusterID;nsid=181322782;c=1676307215066), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-13 16:53:37,442 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcc26de3b869903fd: Processing first storage report for DS-7bc0c30e-8059-4b4a-b749-3bfcf9352799 from datanode 71870c0e-015e-4301-a526-b4af394d64df 2023-02-13 16:53:37,442 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcc26de3b869903fd: from storage DS-7bc0c30e-8059-4b4a-b749-3bfcf9352799 node DatanodeRegistration(127.0.0.1:44175, datanodeUuid=71870c0e-015e-4301-a526-b4af394d64df, infoPort=43515, infoSecurePort=0, ipcPort=36833, storageInfo=lv=-57;cid=testClusterID;nsid=181322782;c=1676307215066), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 16:53:37,522 DEBUG [Listener at localhost/36833] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd 2023-02-13 16:53:37,591 INFO [Listener at localhost/36833] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/zookeeper_0, clientPort=64619, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 16:53:37,604 INFO [Listener at localhost/36833] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64619 2023-02-13 16:53:37,614 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:37,617 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:38,280 INFO [Listener at localhost/36833] util.FSUtils(479): Created version file at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3 with version=8 2023-02-13 16:53:38,280 INFO [Listener at localhost/36833] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/hbase-staging 2023-02-13 16:53:38,576 INFO [Listener at localhost/36833] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 16:53:39,110 INFO [Listener at localhost/36833] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-13 16:53:39,142 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:53:39,142 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 16:53:39,143 INFO [Listener at localhost/36833] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 16:53:39,143 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:53:39,143 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 16:53:39,320 INFO [Listener at localhost/36833] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 16:53:39,396 DEBUG [Listener at localhost/36833] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 16:53:39,488 INFO [Listener at localhost/36833] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38407 2023-02-13 16:53:39,499 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:39,501 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:39,521 INFO [Listener at localhost/36833] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38407 connecting to ZooKeeper ensemble=127.0.0.1:64619 2023-02-13 16:53:39,569 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:384070x0, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 16:53:39,572 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:38407-0x1004e9a266d0000 connected 2023-02-13 16:53:39,606 DEBUG [Listener at localhost/36833] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:53:39,606 DEBUG [Listener at localhost/36833] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:53:39,610 DEBUG [Listener at localhost/36833] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 16:53:39,619 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38407 2023-02-13 16:53:39,619 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38407 2023-02-13 16:53:39,619 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38407 2023-02-13 16:53:39,620 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38407 2023-02-13 16:53:39,620 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38407 2023-02-13 16:53:39,626 INFO [Listener at localhost/36833] master.HMaster(439): hbase.rootdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3, hbase.cluster.distributed=false 2023-02-13 16:53:39,708 INFO [Listener at localhost/36833] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-13 16:53:39,709 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:53:39,709 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 16:53:39,709 INFO [Listener at localhost/36833] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 16:53:39,709 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 16:53:39,709 INFO [Listener at localhost/36833] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 16:53:39,714 INFO [Listener at localhost/36833] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 16:53:39,717 INFO [Listener at localhost/36833] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33907 2023-02-13 16:53:39,719 INFO [Listener at localhost/36833] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 16:53:39,725 DEBUG [Listener at localhost/36833] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 16:53:39,726 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:39,729 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:39,731 INFO [Listener at localhost/36833] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33907 connecting to ZooKeeper ensemble=127.0.0.1:64619 2023-02-13 16:53:39,734 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:339070x0, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 16:53:39,735 DEBUG [Listener at localhost/36833] zookeeper.ZKUtil(164): regionserver:339070x0, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:53:39,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33907-0x1004e9a266d0001 connected 2023-02-13 16:53:39,736 DEBUG [Listener at localhost/36833] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:53:39,737 DEBUG [Listener at localhost/36833] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 16:53:39,740 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33907 2023-02-13 16:53:39,740 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33907 2023-02-13 16:53:39,741 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33907 2023-02-13 16:53:39,742 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33907 2023-02-13 16:53:39,742 DEBUG [Listener at localhost/36833] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33907 2023-02-13 16:53:39,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:39,757 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 16:53:39,758 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:39,777 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 16:53:39,777 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:39,777 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 16:53:39,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 16:53:39,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38407,1676307218418 from backup master directory 2023-02-13 16:53:39,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 16:53:39,783 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:39,783 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 16:53:39,784 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 16:53:39,784 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:39,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 16:53:39,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 16:53:39,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/hbase.id with ID: ef07c3dd-8290-4108-81ac-7402ba210cab 2023-02-13 16:53:39,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:39,935 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:39,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2412281b to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:53:40,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f14246, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:53:40,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 16:53:40,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 16:53:40,058 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 16:53:40,058 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 16:53:40,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:53:40,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:53:40,065 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:53:40,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store-tmp 2023-02-13 16:53:40,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:40,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 16:53:40,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:53:40,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:53:40,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 16:53:40,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:53:40,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:53:40,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676307220129Disabling compacts and flushes for region at 1676307220130 (+1 ms)Disabling writes for close at 1676307220130Writing region close event to WAL at 1676307220130Closed at 1676307220130 2023-02-13 16:53:40,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/WALs/jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:40,153 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38407%2C1676307218418, suffix=, logDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/WALs/jenkins-hbase4.apache.org,38407,1676307218418, archiveDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/oldWALs, maxLogs=10 2023-02-13 16:53:40,209 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44175,DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b,DISK] 2023-02-13 16:53:40,218 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 16:53:40,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/WALs/jenkins-hbase4.apache.org,38407,1676307218418/jenkins-hbase4.apache.org%2C38407%2C1676307218418.1676307220165 2023-02-13 16:53:40,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44175,DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b,DISK]] 2023-02-13 16:53:40,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:53:40,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:40,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:53:40,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:53:40,356 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:53:40,364 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 16:53:40,387 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 16:53:40,401 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:40,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:53:40,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:53:40,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 16:53:40,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:53:40,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68124964, jitterRate=0.015141069889068604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:53:40,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676307220322Initializing all the Stores at 1676307220324 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307220325 (+1 ms)Cleaning up temporary data from old regions at 1676307220415 (+90 ms)Cleaning up detritus from prior splits at 1676307220415Region opened successfully at 1676307220431 (+16 ms) 2023-02-13 16:53:40,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 16:53:40,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 16:53:40,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 16:53:40,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 16:53:40,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 16:53:40,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-02-13 16:53:40,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 16:53:40,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 16:53:40,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 16:53:40,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 16:53:40,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 16:53:40,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 16:53:40,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 16:53:40,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 16:53:40,569 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:40,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 16:53:40,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 16:53:40,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 16:53:40,589 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 16:53:40,589 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 16:53:40,589 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:40,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,38407,1676307218418, sessionid=0x1004e9a266d0000, setting cluster-up flag (Was=false) 2023-02-13 16:53:40,608 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:40,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 16:53:40,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:40,622 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:40,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 16:53:40,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:40,647 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp 2023-02-13 16:53:40,748 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(952): ClusterId : ef07c3dd-8290-4108-81ac-7402ba210cab 2023-02-13 16:53:40,754 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 16:53:40,761 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 16:53:40,761 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 16:53:40,765 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 16:53:40,766 DEBUG [RS:0;jenkins-hbase4:33907] zookeeper.ReadOnlyZKClient(139): Connect 0x3f077a56 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:53:40,775 DEBUG [RS:0;jenkins-hbase4:33907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@31c093c6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:53:40,776 DEBUG [RS:0;jenkins-hbase4:33907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35ff1126, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 16:53:40,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 16:53:40,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:40,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676307250799 2023-02-13 16:53:40,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 16:53:40,810 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 16:53:40,810 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33907 2023-02-13 16:53:40,810 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 16:53:40,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 16:53:40,815 INFO [RS:0;jenkins-hbase4:33907] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 16:53:40,815 INFO [RS:0;jenkins-hbase4:33907] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 16:53:40,815 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 16:53:40,817 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 16:53:40,818 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38407,1676307218418 with isa=jenkins-hbase4.apache.org/172.31.14.131:33907, startcode=1676307219708 2023-02-13 16:53:40,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 16:53:40,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 16:53:40,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 16:53:40,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 16:53:40,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:40,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 16:53:40,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 16:53:40,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 16:53:40,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 16:53:40,848 DEBUG [RS:0;jenkins-hbase4:33907] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 16:53:40,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 16:53:40,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676307220850,5,FailOnTimeoutGroup] 2023-02-13 16:53:40,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676307220852,5,FailOnTimeoutGroup] 2023-02-13 16:53:40,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:40,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 16:53:40,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:40,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:40,869 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 16:53:40,870 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 16:53:40,871 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3 2023-02-13 16:53:40,910 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33643, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 16:53:40,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:40,943 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3 2023-02-13 16:53:40,943 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:45789 2023-02-13 16:53:40,943 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 16:53:40,949 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:53:40,950 DEBUG [RS:0;jenkins-hbase4:33907] zookeeper.ZKUtil(162): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:40,950 WARN [RS:0;jenkins-hbase4:33907] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 16:53:40,950 INFO [RS:0;jenkins-hbase4:33907] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:53:40,951 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1947): logDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/WALs/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:40,953 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33907,1676307219708] 2023-02-13 16:53:40,961 DEBUG [RS:0;jenkins-hbase4:33907] zookeeper.ZKUtil(162): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:40,971 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 16:53:40,980 INFO [RS:0;jenkins-hbase4:33907] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 16:53:41,005 INFO [RS:0;jenkins-hbase4:33907] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 16:53:41,009 INFO [RS:0;jenkins-hbase4:33907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:53:41,009 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,011 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 16:53:41,018 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,019 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,019 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,019 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,019 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,020 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,020 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 16:53:41,020 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,020 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,020 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,020 DEBUG [RS:0;jenkins-hbase4:33907] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 16:53:41,021 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,021 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,021 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,043 INFO [RS:0;jenkins-hbase4:33907] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 16:53:41,046 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33907,1676307219708-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,064 INFO [RS:0;jenkins-hbase4:33907] regionserver.Replication(203): jenkins-hbase4.apache.org,33907,1676307219708 started 2023-02-13 16:53:41,064 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33907,1676307219708, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33907, sessionid=0x1004e9a266d0001 2023-02-13 16:53:41,064 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 16:53:41,064 DEBUG [RS:0;jenkins-hbase4:33907] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:41,064 DEBUG [RS:0;jenkins-hbase4:33907] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33907,1676307219708' 2023-02-13 16:53:41,065 DEBUG [RS:0;jenkins-hbase4:33907] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:41,065 DEBUG [RS:0;jenkins-hbase4:33907] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:41,066 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 16:53:41,066 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 16:53:41,066 DEBUG [RS:0;jenkins-hbase4:33907] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:41,066 DEBUG [RS:0;jenkins-hbase4:33907] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33907,1676307219708' 2023-02-13 16:53:41,066 DEBUG [RS:0;jenkins-hbase4:33907] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 16:53:41,067 DEBUG [RS:0;jenkins-hbase4:33907] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 16:53:41,067 DEBUG [RS:0;jenkins-hbase4:33907] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 16:53:41,067 INFO [RS:0;jenkins-hbase4:33907] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 16:53:41,067 INFO [RS:0;jenkins-hbase4:33907] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 16:53:41,181 INFO [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33907%2C1676307219708, suffix=, logDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/WALs/jenkins-hbase4.apache.org,33907,1676307219708, archiveDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/oldWALs, maxLogs=32 2023-02-13 16:53:41,207 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44175,DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b,DISK] 2023-02-13 16:53:41,216 INFO [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/WALs/jenkins-hbase4.apache.org,33907,1676307219708/jenkins-hbase4.apache.org%2C33907%2C1676307219708.1676307221184 2023-02-13 16:53:41,217 DEBUG [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44175,DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b,DISK]] 2023-02-13 16:53:41,288 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:41,291 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 16:53:41,294 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/info 2023-02-13 16:53:41,295 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 16:53:41,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:41,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 16:53:41,300 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/rep_barrier 2023-02-13 16:53:41,301 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 16:53:41,302 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:41,303 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 16:53:41,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/table 2023-02-13 16:53:41,309 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 16:53:41,310 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:41,312 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740 2023-02-13 16:53:41,313 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740 2023-02-13 16:53:41,318 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 16:53:41,320 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 16:53:41,325 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:53:41,326 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67151334, jitterRate=6.3285231590271E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 16:53:41,326 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676307221288Initializing all the Stores at 1676307221290 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307221291 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307221291Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307221291Cleaning up temporary data from old regions at 1676307221315 (+24 ms)Cleaning up detritus from prior splits at 1676307221316 (+1 ms)Region opened successfully at 1676307221326 (+10 ms) 2023-02-13 16:53:41,327 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 16:53:41,327 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 16:53:41,327 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 16:53:41,327 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 16:53:41,327 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 16:53:41,328 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 16:53:41,328 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676307221327Disabling compacts and flushes for region at 1676307221327Disabling writes for close at 1676307221327Writing region close event to WAL at 1676307221328 (+1 ms)Closed at 1676307221328 2023-02-13 16:53:41,336 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 16:53:41,336 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 16:53:41,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 16:53:41,369 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 16:53:41,372 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 16:53:41,524 DEBUG [jenkins-hbase4:38407] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-13 16:53:41,527 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33907,1676307219708, state=OPENING 2023-02-13 16:53:41,534 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 16:53:41,536 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:41,537 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 16:53:41,539 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:53:41,722 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:41,725 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:53:41,728 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40190, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:53:41,739 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 16:53:41,739 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 16:53:41,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33907%2C1676307219708.meta, suffix=.meta, logDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/WALs/jenkins-hbase4.apache.org,33907,1676307219708, archiveDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/oldWALs, maxLogs=32 2023-02-13 16:53:41,759 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44175,DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b,DISK] 2023-02-13 16:53:41,763 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/WALs/jenkins-hbase4.apache.org,33907,1676307219708/jenkins-hbase4.apache.org%2C33907%2C1676307219708.meta.1676307221743.meta 2023-02-13 16:53:41,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44175,DS-56dc1ecd-c4a5-4fa0-bafb-9659a499b13b,DISK]] 2023-02-13 16:53:41,764 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:53:41,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 16:53:41,783 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 16:53:41,789 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 16:53:41,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 16:53:41,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:41,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 16:53:41,795 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 16:53:41,798 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 16:53:41,800 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/info 2023-02-13 16:53:41,800 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/info 2023-02-13 16:53:41,801 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 16:53:41,802 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:41,802 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 16:53:41,804 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/rep_barrier 2023-02-13 16:53:41,804 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/rep_barrier 2023-02-13 16:53:41,805 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 16:53:41,806 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:41,806 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 16:53:41,807 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/table 2023-02-13 16:53:41,807 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/table 2023-02-13 16:53:41,808 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 16:53:41,809 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:41,811 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740 2023-02-13 16:53:41,814 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740 2023-02-13 16:53:41,818 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 16:53:41,820 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 16:53:41,822 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66146531, jitterRate=-0.014339879155158997}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 16:53:41,822 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676307221795Writing region info on filesystem at 1676307221796 (+1 ms)Initializing all the Stores at 1676307221798 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307221798Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676307221798Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307221798Cleaning up temporary data from old regions at 1676307221815 (+17 ms)Cleaning up detritus from prior splits at 1676307221816 (+1 ms)Running coprocessor post-open hooks at 1676307221822 (+6 ms)Region opened successfully at 1676307221822 2023-02-13 16:53:41,832 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676307221715 2023-02-13 16:53:41,849 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 16:53:41,850 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 16:53:41,850 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33907,1676307219708, state=OPEN 2023-02-13 16:53:41,853 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 16:53:41,853 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 16:53:41,860 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 16:53:41,860 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33907,1676307219708 in 314 msec 2023-02-13 16:53:41,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 16:53:41,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 515 msec 2023-02-13 16:53:41,875 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.1710 sec 2023-02-13 16:53:41,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676307219773, completionTime=-1 2023-02-13 16:53:41,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-13 16:53:41,876 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 16:53:41,942 DEBUG [hconnection-0x1638776d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:53:41,945 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40192, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:53:41,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-13 16:53:41,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676307281963 2023-02-13 16:53:41,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676307341963 2023-02-13 16:53:41,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 87 msec 2023-02-13 16:53:41,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38407,1676307218418-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38407,1676307218418-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38407,1676307218418-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38407, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 16:53:41,998 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 16:53:42,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 16:53:42,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 16:53:42,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 16:53:42,018 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:53:42,021 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:53:42,047 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,049 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221 empty. 2023-02-13 16:53:42,050 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,050 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 16:53:42,072 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 16:53:42,075 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8df3bdbebcbb3c1db6bcb4b3d768f221, NAME => 'hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:53:42,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:42,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 8df3bdbebcbb3c1db6bcb4b3d768f221, disabling compactions & flushes 2023-02-13 16:53:42,090 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. after waiting 0 ms 2023-02-13 16:53:42,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,090 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,090 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 8df3bdbebcbb3c1db6bcb4b3d768f221: Waiting for close lock at 1676307222090Disabling compacts and flushes for region at 1676307222090Disabling writes for close at 1676307222090Writing region close event to WAL at 1676307222090Closed at 1676307222090 2023-02-13 16:53:42,095 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:53:42,112 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676307222098"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307222098"}]},"ts":"1676307222098"} 2023-02-13 16:53:42,141 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:53:42,143 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:53:42,149 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307222143"}]},"ts":"1676307222143"} 2023-02-13 16:53:42,153 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 16:53:42,168 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8df3bdbebcbb3c1db6bcb4b3d768f221, ASSIGN}] 2023-02-13 16:53:42,171 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8df3bdbebcbb3c1db6bcb4b3d768f221, ASSIGN 2023-02-13 16:53:42,173 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8df3bdbebcbb3c1db6bcb4b3d768f221, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:53:42,324 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8df3bdbebcbb3c1db6bcb4b3d768f221, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:42,325 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676307222324"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307222324"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307222324"}]},"ts":"1676307222324"} 2023-02-13 16:53:42,330 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 8df3bdbebcbb3c1db6bcb4b3d768f221, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:53:42,490 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,491 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8df3bdbebcbb3c1db6bcb4b3d768f221, NAME => 'hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:53:42,492 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,492 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:42,493 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,493 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,496 INFO [StoreOpener-8df3bdbebcbb3c1db6bcb4b3d768f221-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,498 DEBUG [StoreOpener-8df3bdbebcbb3c1db6bcb4b3d768f221-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/info 2023-02-13 16:53:42,498 DEBUG [StoreOpener-8df3bdbebcbb3c1db6bcb4b3d768f221-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/info 2023-02-13 16:53:42,498 INFO [StoreOpener-8df3bdbebcbb3c1db6bcb4b3d768f221-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8df3bdbebcbb3c1db6bcb4b3d768f221 columnFamilyName info 2023-02-13 16:53:42,499 INFO [StoreOpener-8df3bdbebcbb3c1db6bcb4b3d768f221-1] regionserver.HStore(310): Store=8df3bdbebcbb3c1db6bcb4b3d768f221/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:42,501 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,502 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,507 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:53:42,510 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:53:42,511 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8df3bdbebcbb3c1db6bcb4b3d768f221; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67799126, jitterRate=0.010285705327987671}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:53:42,511 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8df3bdbebcbb3c1db6bcb4b3d768f221: Running coprocessor pre-open hook at 1676307222493Writing region info on filesystem at 1676307222493Initializing all the Stores at 1676307222495 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676307222495Cleaning up temporary data from old regions at 1676307222503 (+8 ms)Cleaning up detritus from prior splits at 1676307222503Running coprocessor post-open hooks at 1676307222511 (+8 ms)Region opened successfully at 1676307222511 2023-02-13 16:53:42,513 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221., pid=6, masterSystemTime=1676307222484 2023-02-13 16:53:42,518 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,518 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:53:42,519 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8df3bdbebcbb3c1db6bcb4b3d768f221, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:42,520 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676307222519"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307222519"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307222519"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307222519"}]},"ts":"1676307222519"} 2023-02-13 16:53:42,526 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 16:53:42,527 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 8df3bdbebcbb3c1db6bcb4b3d768f221, server=jenkins-hbase4.apache.org,33907,1676307219708 in 193 msec 2023-02-13 16:53:42,530 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 16:53:42,530 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8df3bdbebcbb3c1db6bcb4b3d768f221, ASSIGN in 359 msec 2023-02-13 16:53:42,531 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:53:42,532 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307222532"}]},"ts":"1676307222532"} 2023-02-13 16:53:42,535 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 16:53:42,539 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:53:42,541 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 533 msec 2023-02-13 16:53:42,618 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 16:53:42,620 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:53:42,620 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:53:42,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 16:53:42,675 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:53:42,681 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-02-13 16:53:42,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 16:53:42,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:53:42,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-13 16:53:42,718 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 16:53:42,721 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 16:53:42,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.937sec 2023-02-13 16:53:42,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 16:53:42,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 16:53:42,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 16:53:42,726 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38407,1676307218418-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 16:53:42,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38407,1676307218418-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 16:53:42,740 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 16:53:42,754 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(139): Connect 0x0629018b to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:53:42,760 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c1e7f16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:53:42,778 DEBUG [hconnection-0x77bc4ff1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:53:42,793 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40208, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:53:42,806 INFO [Listener at localhost/36833] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:53:42,807 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(139): Connect 0x7defec53 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:53:42,814 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64b944ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:53:42,814 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:53:42,837 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=412, ProcessCount=168, AvailableMemoryMB=1765 2023-02-13 16:53:42,848 DEBUG [Listener at localhost/36833] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:53:42,853 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59720, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:53:42,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 16:53:42,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:42,872 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:53:42,875 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:53:42,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-13 16:53:42,878 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:42,879 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab empty. 2023-02-13 16:53:42,881 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:42,881 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-13 16:53:42,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 16:53:42,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 16:53:43,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 16:53:43,299 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-13 16:53:43,301 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => f38f1642b70af5caecff7a16ee5e02ab, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:53:43,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 16:53:43,715 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:43,716 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing f38f1642b70af5caecff7a16ee5e02ab, disabling compactions & flushes 2023-02-13 16:53:43,716 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:43,716 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:43,716 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. after waiting 0 ms 2023-02-13 16:53:43,716 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:43,716 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:43,716 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for f38f1642b70af5caecff7a16ee5e02ab: Waiting for close lock at 1676307223716Disabling compacts and flushes for region at 1676307223716Disabling writes for close at 1676307223716Writing region close event to WAL at 1676307223716Closed at 1676307223716 2023-02-13 16:53:43,721 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:53:43,722 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307223722"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307223722"}]},"ts":"1676307223722"} 2023-02-13 16:53:43,725 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:53:43,727 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:53:43,727 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307223727"}]},"ts":"1676307223727"} 2023-02-13 16:53:43,729 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-13 16:53:43,734 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, ASSIGN}] 2023-02-13 16:53:43,735 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, ASSIGN 2023-02-13 16:53:43,737 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:53:43,888 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f38f1642b70af5caecff7a16ee5e02ab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:43,889 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307223888"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307223888"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307223888"}]},"ts":"1676307223888"} 2023-02-13 16:53:43,892 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f38f1642b70af5caecff7a16ee5e02ab, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:53:44,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 16:53:44,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f38f1642b70af5caecff7a16ee5e02ab, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:53:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,055 INFO [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,057 DEBUG [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:44,057 DEBUG [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:44,060 INFO [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f38f1642b70af5caecff7a16ee5e02ab columnFamilyName family1 2023-02-13 16:53:44,063 INFO [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] regionserver.HStore(310): Store=f38f1642b70af5caecff7a16ee5e02ab/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:44,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:44,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:53:44,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f38f1642b70af5caecff7a16ee5e02ab; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74061495, jitterRate=0.10360227525234222}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:53:44,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f38f1642b70af5caecff7a16ee5e02ab: Running coprocessor pre-open hook at 1676307224052Writing region info on filesystem at 1676307224052Initializing all the Stores at 1676307224053 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676307224053Cleaning up temporary data from old regions at 1676307224067 (+14 ms)Cleaning up detritus from prior splits at 1676307224067Running coprocessor post-open hooks at 1676307224075 (+8 ms)Region opened successfully at 1676307224075 2023-02-13 16:53:44,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., pid=11, masterSystemTime=1676307224045 2023-02-13 16:53:44,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:44,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:44,080 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f38f1642b70af5caecff7a16ee5e02ab, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:44,081 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307224080"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307224080"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307224080"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307224080"}]},"ts":"1676307224080"} 2023-02-13 16:53:44,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 16:53:44,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f38f1642b70af5caecff7a16ee5e02ab, server=jenkins-hbase4.apache.org,33907,1676307219708 in 191 msec 2023-02-13 16:53:44,089 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 16:53:44,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, ASSIGN in 353 msec 2023-02-13 16:53:44,091 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:53:44,091 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307224091"}]},"ts":"1676307224091"} 2023-02-13 16:53:44,093 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-13 16:53:44,097 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:53:44,099 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2280 sec 2023-02-13 16:53:45,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 16:53:45,003 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-13 16:53:45,021 DEBUG [hconnection-0x611a5737-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:53:45,024 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49462, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:53:45,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:45,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,203 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:45,203 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:45,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:45,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:45,205 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,205 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,209 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:45,209 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:45,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:45,210 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,210 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:45,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,211 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,213 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:45,213 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:45,214 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:45,215 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:45,215 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:45,215 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:45,215 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:45,216 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:53:45,378 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113042dc3fa93e29486ca70ecd565abe740a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113042dc3fa93e29486ca70ecd565abe740a 2023-02-13 16:53:45,796 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/c637e50b91c1453fa3ccb950c4c570c9 2023-02-13 16:53:45,807 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/c637e50b91c1453fa3ccb950c4c570c9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/c637e50b91c1453fa3ccb950c4c570c9 2023-02-13 16:53:45,815 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/c637e50b91c1453fa3ccb950c4c570c9, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-13 16:53:45,817 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 601ms, sequenceid=5, compaction requested=false 2023-02-13 16:53:45,819 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:45,821 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:45,821 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:45,821 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:45,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:45,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:45,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:45,826 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:45,827 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,827 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:45,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:45,829 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:45,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74bdfea[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:45,829 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,831 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:45,831 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:45,831 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,831 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:45,831 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:45,833 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:45,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:45,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:45,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:45,837 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:45,837 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:45,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:45,837 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:45,837 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,837 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:45,839 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,839 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:45,839 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:45,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:45,840 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,842 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,854 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,854 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,854 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:45,854 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:45,854 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:45,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:45,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:45,854 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:45,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:45,854 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,855 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,855 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,855 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:45,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:45,856 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:45,857 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:45,957 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:45,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:45,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:45,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:45,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,975 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:45,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:45,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:45,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:45,976 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,976 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,978 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:45,978 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:45,978 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:45,978 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:45,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:45,980 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:45,980 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:45,980 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:45,981 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:45,981 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:45,981 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:45,981 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:45,981 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:53:46,008 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c23bb0bf75584adebdf3ac26a9786456 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c23bb0bf75584adebdf3ac26a9786456 2023-02-13 16:53:46,418 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/d9a2b16707e445eda2d680fc0f041ab5 2023-02-13 16:53:46,426 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/d9a2b16707e445eda2d680fc0f041ab5 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/d9a2b16707e445eda2d680fc0f041ab5 2023-02-13 16:53:46,434 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/d9a2b16707e445eda2d680fc0f041ab5, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:53:46,436 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 455ms, sequenceid=9, compaction requested=false 2023-02-13 16:53:46,436 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:46,436 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:46,436 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:46,439 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,439 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:46,439 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,439 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:46,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:46,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ccfa71a[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:46,441 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:46,441 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,442 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,443 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:46,443 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:46,443 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:46,443 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,443 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:46,445 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:46,445 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:46,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:46,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:46,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:46,449 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:46,449 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:46,449 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:46,449 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,449 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:46,453 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,453 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,453 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:46,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:46,454 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:46,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,456 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:46,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:46,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:46,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:46,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:46,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:46,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:46,460 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:46,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:46,460 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,460 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,460 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,460 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:46,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:46,460 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:46,461 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:46,561 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:46,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:46,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:46,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:46,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,573 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:46,573 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:46,574 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:46,574 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:46,574 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,574 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,576 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:46,576 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:46,576 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:46,576 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:46,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:46,578 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:46,578 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:46,578 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:46,578 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:46,578 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:46,578 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:46,578 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:46,578 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:47,004 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010b6bd5ede25649389e6e59959b93ff5f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512010b6bd5ede25649389e6e59959b93ff5f 2023-02-13 16:53:47,313 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:53:47,373 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 16:53:47,374 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 16:53:47,417 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/cea2a2f1064641a7ba8645530002521e 2023-02-13 16:53:47,428 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/cea2a2f1064641a7ba8645530002521e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/cea2a2f1064641a7ba8645530002521e 2023-02-13 16:53:47,439 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/cea2a2f1064641a7ba8645530002521e, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 16:53:47,441 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 863ms, sequenceid=13, compaction requested=true 2023-02-13 16:53:47,441 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:47,441 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:47,441 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:47,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:47,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:47,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:47,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:47,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:47,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:47,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:47,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:47,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:47,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@166f12b4[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:47,462 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:47,462 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,463 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,463 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:47,463 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:47,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:47,464 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,464 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:47,466 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:47,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:47,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:47,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:47,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:47,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:47,471 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:47,471 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:47,471 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:47,471 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,471 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:47,473 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,473 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,473 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:47,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:47,474 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,477 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:47,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,482 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:47,482 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:47,482 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:47,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:47,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:47,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:47,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:47,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:47,484 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:47,484 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:47,584 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:47,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:47,594 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:47,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:47,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,600 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:47,600 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:47,600 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:47,600 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:47,601 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,601 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,602 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,602 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:47,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:47,603 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:47,603 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:47,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:47,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:47,604 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:47,604 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:47,605 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:47,605 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:47,605 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:47,605 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:47,605 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:47,605 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:48,031 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205110bf8e6dc8a4aa688e578f4ea86140d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205110bf8e6dc8a4aa688e578f4ea86140d 2023-02-13 16:53:48,444 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/80a474a0b8e24d29938c701e778bac5c 2023-02-13 16:53:48,452 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/80a474a0b8e24d29938c701e778bac5c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/80a474a0b8e24d29938c701e778bac5c 2023-02-13 16:53:48,462 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/80a474a0b8e24d29938c701e778bac5c, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 16:53:48,463 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 858ms, sequenceid=17, compaction requested=true 2023-02-13 16:53:48,464 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:48,464 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:48,467 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:48,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:48,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:48,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:48,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:48,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@333bf320[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:48,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:48,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,472 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:48,473 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:48,473 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,473 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:48,473 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:48,475 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,475 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:48,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:48,475 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:48,475 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:48,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:48,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:48,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:48,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:48,479 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:48,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:48,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:48,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,479 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:48,481 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,481 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,481 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:48,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:48,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:48,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:48,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:48,482 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:48,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:48,483 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:48,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,488 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,488 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:48,488 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:48,488 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:48,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:48,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:48,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:48,488 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:48,488 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,488 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,489 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:48,489 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,489 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,489 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:48,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:48,489 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:48,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:48,570 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 16:53:48,570 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 16:53:48,589 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:48,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:48,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:48,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:48,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,602 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:48,602 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:48,603 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:48,603 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:48,603 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,603 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,605 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:48,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:48,605 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,606 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:48,606 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:48,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:48,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:48,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:48,607 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:48,607 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:48,607 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:48,608 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:48,608 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:48,608 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:48,608 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:48,608 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:49,040 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283a49c83a10d54280bdd928123fc8c273 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283a49c83a10d54280bdd928123fc8c273 2023-02-13 16:53:49,058 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/75fb21089d4d408da73b26eecc1b8943 2023-02-13 16:53:49,070 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/75fb21089d4d408da73b26eecc1b8943 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/75fb21089d4d408da73b26eecc1b8943 2023-02-13 16:53:49,080 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/75fb21089d4d408da73b26eecc1b8943, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 16:53:49,081 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 473ms, sequenceid=21, compaction requested=true 2023-02-13 16:53:49,081 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:49,081 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:49,081 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:49,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:49,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:49,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:49,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,089 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:49,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:49,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,090 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:49,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:49,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:49,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:49,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:49,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ad6ce22[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:49,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:49,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:49,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:49,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:49,098 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,098 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:49,098 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:49,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:49,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:49,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:49,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:49,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:49,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:49,103 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:49,103 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:49,103 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:49,103 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,104 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:49,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:49,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:49,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:49,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:49,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:49,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:49,107 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,110 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:49,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:49,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:49,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:49,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:49,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:49,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:49,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:49,116 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,116 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,116 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:49,116 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:49,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:49,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:49,217 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:49,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:49,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:49,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:49,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,228 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:49,228 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:49,229 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:49,229 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:49,229 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,229 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,231 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,231 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:49,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:49,231 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:49,231 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,232 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:49,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:49,233 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:49,233 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:49,233 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:49,233 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:49,233 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:49,233 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:49,233 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:49,234 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:49,656 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231764cf83d37ce40c9a6cfe1e022bec33b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231764cf83d37ce40c9a6cfe1e022bec33b 2023-02-13 16:53:50,066 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/f3dd247864e44a8380df638421e5e59d 2023-02-13 16:53:50,075 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/f3dd247864e44a8380df638421e5e59d as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/f3dd247864e44a8380df638421e5e59d 2023-02-13 16:53:50,083 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/f3dd247864e44a8380df638421e5e59d, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 16:53:50,084 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 850ms, sequenceid=25, compaction requested=true 2023-02-13 16:53:50,085 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:50,085 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:50,085 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:50,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:50,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:50,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:50,088 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,088 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:50,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:50,088 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,088 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:50,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:50,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:50,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:50,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:50,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b304315[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:50,091 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:50,091 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,093 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,093 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:50,093 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:50,093 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:50,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:50,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:50,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:50,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:50,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:50,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:50,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:50,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:50,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:50,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:50,099 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:50,099 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:50,099 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:50,099 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,099 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:50,101 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,101 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:50,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:50,101 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:50,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:50,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:50,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:50,102 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:50,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,104 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:50,107 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:50,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:50,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:50,107 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:50,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:50,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:50,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:50,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,108 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:50,108 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:50,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:50,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:50,208 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:50,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:50,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:50,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:50,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,219 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:50,219 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:50,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:50,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:50,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,222 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:50,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:50,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:50,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:50,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:50,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:50,224 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:50,225 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:50,225 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:50,646 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101537597a82b5447d0a37f10208623a162 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101537597a82b5447d0a37f10208623a162 2023-02-13 16:53:51,055 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/9e45e1567d56433989d9d554ba6bb1e3 2023-02-13 16:53:51,065 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/9e45e1567d56433989d9d554ba6bb1e3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9e45e1567d56433989d9d554ba6bb1e3 2023-02-13 16:53:51,073 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9e45e1567d56433989d9d554ba6bb1e3, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 16:53:51,075 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 850ms, sequenceid=29, compaction requested=true 2023-02-13 16:53:51,075 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:51,075 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:51,075 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:51,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:51,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:51,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:51,079 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:51,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:51,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:51,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15a40475[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:51,081 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:51,081 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,083 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,083 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:51,083 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:51,083 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:51,083 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,083 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:51,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:51,086 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:51,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:51,086 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:51,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:51,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:51,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:51,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:51,090 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:51,091 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:51,091 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:51,091 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,091 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:51,093 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,093 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,093 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:51,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,095 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:51,106 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:51,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:51,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:51,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:51,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:51,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:51,106 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,107 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:51,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:51,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:51,207 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:51,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:51,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:51,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:51,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,220 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:51,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:51,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:51,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:51,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,223 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:51,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:51,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:51,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:51,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:51,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:51,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,228 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:51,228 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:51,229 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:51,229 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:51,229 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:51,229 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:51,229 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:51,229 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:51,657 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fb3a7c1a996c46c79ef7d9e7964937c9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103fb3a7c1a996c46c79ef7d9e7964937c9 2023-02-13 16:53:51,666 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/4449c454001e47a795df02e8dab5e701 2023-02-13 16:53:51,673 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/4449c454001e47a795df02e8dab5e701 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/4449c454001e47a795df02e8dab5e701 2023-02-13 16:53:51,679 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/4449c454001e47a795df02e8dab5e701, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 16:53:51,681 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 452ms, sequenceid=33, compaction requested=true 2023-02-13 16:53:51,681 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:51,681 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:51,681 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:51,682 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:51,682 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,682 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:51,682 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:51,683 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,683 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:51,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:51,684 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,684 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:51,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:51,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:51,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:51,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:51,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79ea5712[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:51,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:51,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,690 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,690 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:51,690 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:51,690 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:51,690 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,690 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:51,692 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:51,692 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,692 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:51,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:51,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:51,692 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:51,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:51,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:51,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:51,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:51,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:51,697 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:51,697 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:51,697 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:51,697 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,697 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:51,699 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,699 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:51,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:51,699 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:51,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:51,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:51,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:51,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,701 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:51,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,703 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,706 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,706 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:51,706 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,706 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:51,706 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,707 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,707 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,707 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,707 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:51,706 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:51,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:51,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:51,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:51,708 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:51,708 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:51,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:51,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:51,809 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:51,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:51,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:51,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:51,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,820 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:51,820 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:51,820 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:51,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:51,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,823 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,823 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:51,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:51,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:51,824 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:51,824 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:51,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:51,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:51,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:51,827 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:51,832 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:51,832 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:51,832 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:51,832 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:51,832 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:52,258 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090779d59c4cbdc640aab54dbe97ba45d84a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015090779d59c4cbdc640aab54dbe97ba45d84a 2023-02-13 16:53:52,669 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/ccd79bf43966448f907018b7491c8579 2023-02-13 16:53:52,676 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/ccd79bf43966448f907018b7491c8579 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/ccd79bf43966448f907018b7491c8579 2023-02-13 16:53:52,683 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/ccd79bf43966448f907018b7491c8579, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 16:53:52,684 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 852ms, sequenceid=37, compaction requested=true 2023-02-13 16:53:52,684 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:52,684 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:52,684 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:52,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:52,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:52,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:52,686 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,686 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:52,687 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,687 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:52,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:52,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:52,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@259131d3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:52,688 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:52,688 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,690 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,690 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:52,690 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:52,690 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:52,690 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,690 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:52,691 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:52,691 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,691 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:52,691 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:52,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:52,694 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:52,695 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:52,695 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:52,695 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,695 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:52,697 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,697 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,697 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:52,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:52,698 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,699 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,702 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,702 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:52,702 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:52,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:52,702 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:52,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:52,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:52,702 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,702 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:52,702 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:52,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,703 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:52,703 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:52,803 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:52,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:52,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:52,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:52,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,814 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:52,814 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:52,815 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 16:53:52,815 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 16:53:52,815 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,815 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,817 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:52,817 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:52,817 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:52,817 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:52,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 16:53:52,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:52,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 16:53:52,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:52,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 16:53:52,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:52,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:52,820 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:52,820 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. started... 2023-02-13 16:53:52,820 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f38f1642b70af5caecff7a16ee5e02ab 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:53:53,240 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112019e63ce5dd3c434b97387f3585956935 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112019e63ce5dd3c434b97387f3585956935 2023-02-13 16:53:53,649 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/41d459196b61409fb7b40b888783dc43 2023-02-13 16:53:53,658 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/41d459196b61409fb7b40b888783dc43 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/41d459196b61409fb7b40b888783dc43 2023-02-13 16:53:53,666 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/41d459196b61409fb7b40b888783dc43, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 16:53:53,668 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f38f1642b70af5caecff7a16ee5e02ab in 848ms, sequenceid=41, compaction requested=true 2023-02-13 16:53:53,668 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:53,668 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:53,668 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:53,669 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:53,669 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,669 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 16:53:53,669 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 16:53:53,671 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:53,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:53,671 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:53,672 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,672 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:53,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:53,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:53,673 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 16:53:53,673 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c40f69f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:53,673 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 16:53:53,673 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,674 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,674 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 16:53:53,674 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,675 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:53,674 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 16:53:53,675 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:53,680 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 16:53:53,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:53,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:53,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:53,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 16:53:53,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:53,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:53,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:53,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:53,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 16:53:53,687 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 16:53:53,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:53,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:53,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,687 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:53,689 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,689 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:53,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:53,689 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:53,689 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:53,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:53,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:53,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:53,691 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:53,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,693 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,694 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:53,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 16:53:53,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:53,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:53,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:53,694 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 16:53:53,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:53,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:53,695 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:53,695 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:53,795 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:53,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:53,802 DEBUG [Listener at localhost/36833] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:53:53,805 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51060, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:53:53,806 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:53,821 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 16:53:53,822 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 16:53:53,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:53:53,823 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307833823 2023-02-13 16:53:53,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:53:53,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-13 16:53:53,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:53,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x4fbb4281 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:53:53,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7051cf00, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:53:53,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,774 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103598937bca4ce4731a420674f4a94a96f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103598937bca4ce4731a420674f4a94a96f 2023-02-13 16:53:54,775 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:54,794 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:53:54,795 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45810, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:53:54,806 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:53:54,808 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51064, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:53:54,841 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dba6328a83a74550943270686f7645ea first=Optional[r3] last=Optional[r6] 2023-02-13 16:53:54,855 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dba6328a83a74550943270686f7645ea}] 2023-02-13 16:53:54,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dba6328a83a74550943270686f7645ea for inclusion in f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:54,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 16:53:54,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:53:54,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:54,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dba6328a83a74550943270686f7645ea to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ntj42o4dhl7u0efg53v1ki3a1qcshdmpgahpvlhg7kuc8bj75eappug5lps6d0qq/family1/dba6328a83a74550943270686f7645ea 2023-02-13 16:53:54,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ntj42o4dhl7u0efg53v1ki3a1qcshdmpgahpvlhg7kuc8bj75eappug5lps6d0qq/family1/dba6328a83a74550943270686f7645ea as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_ 2023-02-13 16:53:54,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ntj42o4dhl7u0efg53v1ki3a1qcshdmpgahpvlhg7kuc8bj75eappug5lps6d0qq/family1/dba6328a83a74550943270686f7645ea into f38f1642b70af5caecff7a16ee5e02ab/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_ - updating store file list. 2023-02-13 16:53:54,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_ into f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:54,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ntj42o4dhl7u0efg53v1ki3a1qcshdmpgahpvlhg7kuc8bj75eappug5lps6d0qq/family1/dba6328a83a74550943270686f7645ea into f38f1642b70af5caecff7a16ee5e02ab/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_) 2023-02-13 16:53:54,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ntj42o4dhl7u0efg53v1ki3a1qcshdmpgahpvlhg7kuc8bj75eappug5lps6d0qq/family1/dba6328a83a74550943270686f7645ea 2023-02-13 16:53:54,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:53:54,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:53:54,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): f38f1642b70af5caecff7a16ee5e02ab/family1 is initiating minor compaction 2023-02-13 16:53:54,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@fa21478; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:53:54,901 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f38f1642b70af5caecff7a16ee5e02ab/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:54,902 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/d9a2b16707e445eda2d680fc0f041ab5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/cea2a2f1064641a7ba8645530002521e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/80a474a0b8e24d29938c701e778bac5c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/75fb21089d4d408da73b26eecc1b8943, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/f3dd247864e44a8380df638421e5e59d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9e45e1567d56433989d9d554ba6bb1e3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/4449c454001e47a795df02e8dab5e701, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/ccd79bf43966448f907018b7491c8579, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/41d459196b61409fb7b40b888783dc43, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp, totalSize=50.2 K 2023-02-13 16:53:54,902 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting d9a2b16707e445eda2d680fc0f041ab5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:53:54,902 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting cea2a2f1064641a7ba8645530002521e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:53:54,902 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 80a474a0b8e24d29938c701e778bac5c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:53:54,902 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 75fb21089d4d408da73b26eecc1b8943, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:53:54,903 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting f3dd247864e44a8380df638421e5e59d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:53:54,903 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 9e45e1567d56433989d9d554ba6bb1e3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:53:54,903 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 4449c454001e47a795df02e8dab5e701, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:53:54,903 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting ccd79bf43966448f907018b7491c8579, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:53:54,903 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 41d459196b61409fb7b40b888783dc43, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:53:54,903 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:53:54,915 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:54,917 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:53:54,922 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283a49c83a10d54280bdd928123fc8c273 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283a49c83a10d54280bdd928123fc8c273 2023-02-13 16:53:54,924 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231764cf83d37ce40c9a6cfe1e022bec33b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231764cf83d37ce40c9a6cfe1e022bec33b 2023-02-13 16:53:54,926 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101537597a82b5447d0a37f10208623a162 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101537597a82b5447d0a37f10208623a162 2023-02-13 16:53:54,928 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103fb3a7c1a996c46c79ef7d9e7964937c9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103fb3a7c1a996c46c79ef7d9e7964937c9 2023-02-13 16:53:54,928 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-13 16:53:54,928 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:54,958 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f38f1642b70af5caecff7a16ee5e02ab#family1#compaction#10 average throughput is 0.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:53:55,385 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/8ccf8355e22642ed875b5b3fcd3d0b6e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8ccf8355e22642ed875b5b3fcd3d0b6e 2023-02-13 16:53:55,401 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f38f1642b70af5caecff7a16ee5e02ab/family1 of f38f1642b70af5caecff7a16ee5e02ab into 8ccf8355e22642ed875b5b3fcd3d0b6e(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:53:55,401 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:55,403 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1676307234899; duration=0sec 2023-02-13 16:53:55,404 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:53:55,420 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050dbb65c5d4e74b5da5d080d59a2885d2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050dbb65c5d4e74b5da5d080d59a2885d2 2023-02-13 16:53:55,425 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:55,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/d9a2b16707e445eda2d680fc0f041ab5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/cea2a2f1064641a7ba8645530002521e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/80a474a0b8e24d29938c701e778bac5c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/75fb21089d4d408da73b26eecc1b8943, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/f3dd247864e44a8380df638421e5e59d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9e45e1567d56433989d9d554ba6bb1e3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/4449c454001e47a795df02e8dab5e701, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/ccd79bf43966448f907018b7491c8579, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/41d459196b61409fb7b40b888783dc43, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_] to archive 2023-02-13 16:53:55,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:53:55,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/d9a2b16707e445eda2d680fc0f041ab5 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/d9a2b16707e445eda2d680fc0f041ab5 2023-02-13 16:53:55,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/cea2a2f1064641a7ba8645530002521e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/cea2a2f1064641a7ba8645530002521e 2023-02-13 16:53:55,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/80a474a0b8e24d29938c701e778bac5c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/80a474a0b8e24d29938c701e778bac5c 2023-02-13 16:53:55,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/75fb21089d4d408da73b26eecc1b8943 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/75fb21089d4d408da73b26eecc1b8943 2023-02-13 16:53:55,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/f3dd247864e44a8380df638421e5e59d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/f3dd247864e44a8380df638421e5e59d 2023-02-13 16:53:55,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9e45e1567d56433989d9d554ba6bb1e3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9e45e1567d56433989d9d554ba6bb1e3 2023-02-13 16:53:55,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/4449c454001e47a795df02e8dab5e701 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/4449c454001e47a795df02e8dab5e701 2023-02-13 16:53:55,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/ccd79bf43966448f907018b7491c8579 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/ccd79bf43966448f907018b7491c8579 2023-02-13 16:53:55,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/41d459196b61409fb7b40b888783dc43 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/41d459196b61409fb7b40b888783dc43 2023-02-13 16:53:55,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8e5d8208fd434b28a0261c69e72c78cc_SeqId_44_ 2023-02-13 16:53:55,458 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1c7e5cd62e4a472ab08e5b6323307082 first=Optional[r00] last=Optional[r2] 2023-02-13 16:53:55,459 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1c7e5cd62e4a472ab08e5b6323307082}] 2023-02-13 16:53:55,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1c7e5cd62e4a472ab08e5b6323307082 for inclusion in f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:55,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 16:53:55,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:53:55,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:55,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1c7e5cd62e4a472ab08e5b6323307082 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r6qvl8n3tdvq0qq2mq2j6q35nj60mu2uk7i4l9rmej3cl2453fn5l0qdgulf7iif/family1/1c7e5cd62e4a472ab08e5b6323307082 2023-02-13 16:53:55,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r6qvl8n3tdvq0qq2mq2j6q35nj60mu2uk7i4l9rmej3cl2453fn5l0qdgulf7iif/family1/1c7e5cd62e4a472ab08e5b6323307082 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_ 2023-02-13 16:53:55,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r6qvl8n3tdvq0qq2mq2j6q35nj60mu2uk7i4l9rmej3cl2453fn5l0qdgulf7iif/family1/1c7e5cd62e4a472ab08e5b6323307082 into f38f1642b70af5caecff7a16ee5e02ab/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_ - updating store file list. 2023-02-13 16:53:55,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_ into f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:55,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r6qvl8n3tdvq0qq2mq2j6q35nj60mu2uk7i4l9rmej3cl2453fn5l0qdgulf7iif/family1/1c7e5cd62e4a472ab08e5b6323307082 into f38f1642b70af5caecff7a16ee5e02ab/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_) 2023-02-13 16:53:55,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r6qvl8n3tdvq0qq2mq2j6q35nj60mu2uk7i4l9rmej3cl2453fn5l0qdgulf7iif/family1/1c7e5cd62e4a472ab08e5b6323307082 2023-02-13 16:53:55,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:53:55,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:53:55,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): f38f1642b70af5caecff7a16ee5e02ab/family1 is initiating minor compaction (all files) 2023-02-13 16:53:55,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1756e11e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:53:55,483 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f38f1642b70af5caecff7a16ee5e02ab/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:55,483 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/c637e50b91c1453fa3ccb950c4c570c9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8ccf8355e22642ed875b5b3fcd3d0b6e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp, totalSize=358.6 K 2023-02-13 16:53:55,484 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting c637e50b91c1453fa3ccb950c4c570c9, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842816 2023-02-13 16:53:55,484 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 8ccf8355e22642ed875b5b3fcd3d0b6e, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242816 2023-02-13 16:53:55,485 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842816 2023-02-13 16:53:55,485 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:55,486 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:53:55,488 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113042dc3fa93e29486ca70ecd565abe740a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113042dc3fa93e29486ca70ecd565abe740a 2023-02-13 16:53:55,490 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512010b6bd5ede25649389e6e59959b93ff5f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512010b6bd5ede25649389e6e59959b93ff5f 2023-02-13 16:53:55,492 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205110bf8e6dc8a4aa688e578f4ea86140d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205110bf8e6dc8a4aa688e578f4ea86140d 2023-02-13 16:53:55,492 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-13 16:53:55,492 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 16:53:55,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:55,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:55,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:53:55,494 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 16:53:55,496 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6830 sec 2023-02-13 16:53:55,529 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f38f1642b70af5caecff7a16ee5e02ab#family1#compaction#11 average throughput is 5.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:53:55,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 16:53:55,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:56,078 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-13 16:53:56,080 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-13 16:53:56,080 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-13 16:53:56,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-13 16:53:56,098 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, REOPEN/MOVE}] 2023-02-13 16:53:56,099 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, REOPEN/MOVE 2023-02-13 16:53:56,100 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=f38f1642b70af5caecff7a16ee5e02ab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:56,100 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307236100"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307236100"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307236100"}]},"ts":"1676307236100"} 2023-02-13 16:53:56,102 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure f38f1642b70af5caecff7a16ee5e02ab, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:53:56,261 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f38f1642b70af5caecff7a16ee5e02ab, disabling compactions & flushes 2023-02-13 16:53:56,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,367 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/.tmp/family1/78107d2d29b0430f9c999f4a1ea22b0b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/78107d2d29b0430f9c999f4a1ea22b0b 2023-02-13 16:53:56,374 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f38f1642b70af5caecff7a16ee5e02ab/family1 of f38f1642b70af5caecff7a16ee5e02ab into 78107d2d29b0430f9c999f4a1ea22b0b(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:53:56,374 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:56,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. after waiting 0 ms 2023-02-13 16:53:56,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,374 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1676307235482; duration=0sec 2023-02-13 16:53:56,374 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:53:56,377 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/c637e50b91c1453fa3ccb950c4c570c9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8ccf8355e22642ed875b5b3fcd3d0b6e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_] to archive 2023-02-13 16:53:56,378 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:53:56,380 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/c637e50b91c1453fa3ccb950c4c570c9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/c637e50b91c1453fa3ccb950c4c570c9 2023-02-13 16:53:56,381 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8ccf8355e22642ed875b5b3fcd3d0b6e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/8ccf8355e22642ed875b5b3fcd3d0b6e 2023-02-13 16:53:56,383 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/9dfa4eeeb5d74c17b56ab97f9d6f61e8_SeqId_47_ 2023-02-13 16:53:56,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 16:53:56,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f38f1642b70af5caecff7a16ee5e02ab: Waiting for close lock at 1676307236262Running coprocessor pre-close hooks at 1676307236262Disabling compacts and flushes for region at 1676307236262Disabling writes for close at 1676307236374 (+112 ms)Writing region close event to WAL at 1676307236385 (+11 ms)Running coprocessor post-close hooks at 1676307236389 (+4 ms)Closed at 1676307236390 (+1 ms) 2023-02-13 16:53:56,390 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: f38f1642b70af5caecff7a16ee5e02ab to self. 2023-02-13 16:53:56,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,392 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=f38f1642b70af5caecff7a16ee5e02ab, regionState=CLOSED 2023-02-13 16:53:56,392 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307236392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307236392"}]},"ts":"1676307236392"} 2023-02-13 16:53:56,396 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-13 16:53:56,396 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure f38f1642b70af5caecff7a16ee5e02ab, server=jenkins-hbase4.apache.org,33907,1676307219708 in 292 msec 2023-02-13 16:53:56,397 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=true 2023-02-13 16:53:56,547 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=f38f1642b70af5caecff7a16ee5e02ab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:56,548 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307236547"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307236547"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307236547"}]},"ts":"1676307236547"} 2023-02-13 16:53:56,550 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure f38f1642b70af5caecff7a16ee5e02ab, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:53:56,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f38f1642b70af5caecff7a16ee5e02ab, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:53:56,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:56,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,708 INFO [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,709 DEBUG [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:56,710 DEBUG [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:56,710 INFO [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f38f1642b70af5caecff7a16ee5e02ab columnFamilyName family1 2023-02-13 16:53:56,717 DEBUG [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/78107d2d29b0430f9c999f4a1ea22b0b 2023-02-13 16:53:56,717 INFO [StoreOpener-f38f1642b70af5caecff7a16ee5e02ab-1] regionserver.HStore(310): Store=f38f1642b70af5caecff7a16ee5e02ab/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:56,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:53:56,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f38f1642b70af5caecff7a16ee5e02ab; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68369626, jitterRate=0.01878681778907776}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:53:56,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f38f1642b70af5caecff7a16ee5e02ab: Running coprocessor pre-open hook at 1676307236707Writing region info on filesystem at 1676307236707Initializing all the Stores at 1676307236708 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676307236708Cleaning up temporary data from old regions at 1676307236720 (+12 ms)Cleaning up detritus from prior splits at 1676307236721 (+1 ms)Running coprocessor post-open hooks at 1676307236724 (+3 ms)Region opened successfully at 1676307236724 2023-02-13 16:53:56,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., pid=17, masterSystemTime=1676307236703 2023-02-13 16:53:56,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:53:56,727 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=f38f1642b70af5caecff7a16ee5e02ab, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:56,728 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676307236727"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307236727"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307236727"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307236727"}]},"ts":"1676307236727"} 2023-02-13 16:53:56,732 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 16:53:56,732 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure f38f1642b70af5caecff7a16ee5e02ab, server=jenkins-hbase4.apache.org,33907,1676307219708 in 180 msec 2023-02-13 16:53:56,734 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-13 16:53:56,734 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=f38f1642b70af5caecff7a16ee5e02ab, REOPEN/MOVE in 634 msec 2023-02-13 16:53:56,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 16:53:56,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 651 msec 2023-02-13 16:53:56,740 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0790 sec 2023-02-13 16:53:56,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-13 16:53:56,749 INFO [Listener at localhost/36833] compactions.TestMobCompactor(278): alter status finished 2023-02-13 16:53:56,750 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 16:53:56,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 16:53:56,753 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 16:53:56,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:53:56,754 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307836754 2023-02-13 16:53:56,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:53:56,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-13 16:53:56,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:56,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x3ddb30aa to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:53:56,761 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75b67acf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:53:56,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:56,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:56,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:56,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:56,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:57,600 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289a3bc8fb65ef49558272d8d7dd89104f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289a3bc8fb65ef49558272d8d7dd89104f 2023-02-13 16:53:57,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:57,615 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:53:57,616 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45812, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:53:57,625 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:53:57,627 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51070, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:53:57,640 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f9a1e232cbb426ea50e228a28d1337a first=Optional[r05] last=Optional[r8] 2023-02-13 16:53:57,641 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f9a1e232cbb426ea50e228a28d1337a}] 2023-02-13 16:53:57,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f9a1e232cbb426ea50e228a28d1337a for inclusion in f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:57,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 16:53:57,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:53:57,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f38f1642b70af5caecff7a16ee5e02ab: 2023-02-13 16:53:57,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f9a1e232cbb426ea50e228a28d1337a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dsqbqhsv13pac438r9f217u505q01mtgki0b0j274r85o944fcht2etiu2shiv3v/family1/1f9a1e232cbb426ea50e228a28d1337a 2023-02-13 16:53:57,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dsqbqhsv13pac438r9f217u505q01mtgki0b0j274r85o944fcht2etiu2shiv3v/family1/1f9a1e232cbb426ea50e228a28d1337a as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/42c2484fd4e44e868fc6b11acec428e5_SeqId_53_ 2023-02-13 16:53:57,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dsqbqhsv13pac438r9f217u505q01mtgki0b0j274r85o944fcht2etiu2shiv3v/family1/1f9a1e232cbb426ea50e228a28d1337a into f38f1642b70af5caecff7a16ee5e02ab/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/42c2484fd4e44e868fc6b11acec428e5_SeqId_53_ - updating store file list. 2023-02-13 16:53:57,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/42c2484fd4e44e868fc6b11acec428e5_SeqId_53_ into f38f1642b70af5caecff7a16ee5e02ab/family1 2023-02-13 16:53:57,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dsqbqhsv13pac438r9f217u505q01mtgki0b0j274r85o944fcht2etiu2shiv3v/family1/1f9a1e232cbb426ea50e228a28d1337a into f38f1642b70af5caecff7a16ee5e02ab/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/family1/42c2484fd4e44e868fc6b11acec428e5_SeqId_53_) 2023-02-13 16:53:57,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dsqbqhsv13pac438r9f217u505q01mtgki0b0j274r85o944fcht2etiu2shiv3v/family1/1f9a1e232cbb426ea50e228a28d1337a 2023-02-13 16:53:57,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:57,706 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:53:57,708 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112019e63ce5dd3c434b97387f3585956935 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112019e63ce5dd3c434b97387f3585956935 2023-02-13 16:53:57,710 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c23bb0bf75584adebdf3ac26a9786456 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c23bb0bf75584adebdf3ac26a9786456 2023-02-13 16:53:57,710 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-13 16:53:57,710 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 16:53:57,710 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:57,710 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:57,711 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 16:53:57,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 16:53:57,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:53:57,712 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 16:53:57,714 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 961 msec 2023-02-13 16:53:57,805 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 211) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x4fbb4281 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1479189715.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xf8413f0-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x506fe693-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_889282984_15 at /127.0.0.1:34808 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34770 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xf8413f0-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xf8413f0-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:47432 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34820 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34872 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34924 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:33907-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34756 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x506fe693-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x4fbb4281-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xf8413f0-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:47402 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x3ddb30aa-SendThread(127.0.0.1:64619) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34768 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34886 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xf8413f0-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34774 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34814 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:47476 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x506fe693-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x3ddb30aa-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x3ddb30aa sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1479189715.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x4fbb4281-SendThread(127.0.0.1:64619) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34758 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34874 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=591 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=387 (was 412), ProcessCount=168 (was 168), AvailableMemoryMB=1340 (was 1765) 2023-02-13 16:53:57,818 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=591, MaxFileDescriptor=60000, SystemLoadAverage=387, ProcessCount=168, AvailableMemoryMB=1339 2023-02-13 16:53:57,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 16:53:57,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:57,823 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:53:57,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-13 16:53:57,824 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:53:57,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:53:57,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:57,827 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5 empty. 2023-02-13 16:53:57,827 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:57,827 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-13 16:53:57,840 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 16:53:57,842 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => d9834553e07b975c30d650299cbec8b5, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:53:57,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:53:58,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:53:58,251 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:58,251 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing d9834553e07b975c30d650299cbec8b5, disabling compactions & flushes 2023-02-13 16:53:58,251 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,251 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,251 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. after waiting 0 ms 2023-02-13 16:53:58,251 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,251 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,251 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for d9834553e07b975c30d650299cbec8b5: Waiting for close lock at 1676307238251Disabling compacts and flushes for region at 1676307238251Disabling writes for close at 1676307238251Writing region close event to WAL at 1676307238251Closed at 1676307238251 2023-02-13 16:53:58,254 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:53:58,255 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307238255"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307238255"}]},"ts":"1676307238255"} 2023-02-13 16:53:58,257 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:53:58,258 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:53:58,258 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307238258"}]},"ts":"1676307238258"} 2023-02-13 16:53:58,259 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-13 16:53:58,262 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d9834553e07b975c30d650299cbec8b5, ASSIGN}] 2023-02-13 16:53:58,264 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d9834553e07b975c30d650299cbec8b5, ASSIGN 2023-02-13 16:53:58,265 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d9834553e07b975c30d650299cbec8b5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:53:58,416 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=d9834553e07b975c30d650299cbec8b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:58,416 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307238416"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307238416"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307238416"}]},"ts":"1676307238416"} 2023-02-13 16:53:58,418 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure d9834553e07b975c30d650299cbec8b5, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:53:58,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:53:58,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d9834553e07b975c30d650299cbec8b5, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:53:58,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:53:58,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,575 INFO [StoreOpener-d9834553e07b975c30d650299cbec8b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,577 DEBUG [StoreOpener-d9834553e07b975c30d650299cbec8b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:53:58,577 DEBUG [StoreOpener-d9834553e07b975c30d650299cbec8b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:53:58,578 INFO [StoreOpener-d9834553e07b975c30d650299cbec8b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d9834553e07b975c30d650299cbec8b5 columnFamilyName family1 2023-02-13 16:53:58,579 INFO [StoreOpener-d9834553e07b975c30d650299cbec8b5-1] regionserver.HStore(310): Store=d9834553e07b975c30d650299cbec8b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:53:58,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d9834553e07b975c30d650299cbec8b5 2023-02-13 16:53:58,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:53:58,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d9834553e07b975c30d650299cbec8b5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73495115, jitterRate=0.09516255557537079}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:53:58,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d9834553e07b975c30d650299cbec8b5: Running coprocessor pre-open hook at 1676307238574Writing region info on filesystem at 1676307238574Initializing all the Stores at 1676307238575 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676307238575Cleaning up temporary data from old regions at 1676307238581 (+6 ms)Cleaning up detritus from prior splits at 1676307238581Running coprocessor post-open hooks at 1676307238586 (+5 ms)Region opened successfully at 1676307238586 2023-02-13 16:53:58,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5., pid=21, masterSystemTime=1676307238570 2023-02-13 16:53:58,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,589 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=d9834553e07b975c30d650299cbec8b5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:58,589 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307238589"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307238589"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307238589"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307238589"}]},"ts":"1676307238589"} 2023-02-13 16:53:58,593 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-13 16:53:58,593 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure d9834553e07b975c30d650299cbec8b5, server=jenkins-hbase4.apache.org,33907,1676307219708 in 173 msec 2023-02-13 16:53:58,595 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 16:53:58,595 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d9834553e07b975c30d650299cbec8b5, ASSIGN in 331 msec 2023-02-13 16:53:58,596 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:53:58,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307238596"}]},"ts":"1676307238596"} 2023-02-13 16:53:58,598 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-13 16:53:58,600 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:53:58,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 779 msec 2023-02-13 16:53:58,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 16:53:58,930 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-13 16:53:58,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:58,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,965 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:53:58,965 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:58,965 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:53:58,965 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:53:58,966 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,966 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,967 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:58,967 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:58,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:58,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:58,968 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:58,968 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:58,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:58,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:58,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:53:58,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:58,970 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:53:58,970 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:58,970 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:58,970 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:58,970 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:53:58,970 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:53:59,376 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:53:59,397 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d66f0e34923c430a8640756ab6df6ef0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d66f0e34923c430a8640756ab6df6ef0 2023-02-13 16:53:59,802 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/e2af4eefa9f54610b991c0c0afaf1c98 2023-02-13 16:53:59,808 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/e2af4eefa9f54610b991c0c0afaf1c98 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2af4eefa9f54610b991c0c0afaf1c98 2023-02-13 16:53:59,813 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2af4eefa9f54610b991c0c0afaf1c98, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-13 16:53:59,814 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 844ms, sequenceid=5, compaction requested=false 2023-02-13 16:53:59,815 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:53:59,815 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:59,815 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:53:59,815 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:53:59,815 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:53:59,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:53:59,817 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,817 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:59,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:53:59,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:59,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:59,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:59,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:53:59,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69f2c6ce[Count = 0] remaining members to acquire global barrier 2023-02-13 16:53:59,819 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:53:59,819 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,820 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:53:59,821 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:53:59,821 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:53:59,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:53:59,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:53:59,822 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:59,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:59,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:59,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:53:59,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:59,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:59,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:59,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:53:59,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:53:59,825 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:53:59,825 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:53:59,825 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:53:59,825 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,825 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:53:59,827 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,827 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,827 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:53:59,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:53:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,828 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:59,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:53:59,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:59,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:53:59,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,829 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:53:59,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,832 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,832 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:59,832 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,832 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:53:59,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:59,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:59,832 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:53:59,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:53:59,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:53:59,832 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:53:59,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:53:59,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:53:59,833 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,833 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,833 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,833 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,833 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:53:59,833 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:53:59,933 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:53:59,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:53:59,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:53:59,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:53:59,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,941 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:53:59,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:59,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:53:59,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:53:59,943 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,943 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,944 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:53:59,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:53:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:53:59,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:53:59,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:53:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:53:59,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:53:59,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:53:59,945 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:53:59,945 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:53:59,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:53:59,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:53:59,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:53:59,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:53:59,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:53:59,946 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:54:00,361 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282f5bea6ecfc946f8963149762f9cedad to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511282f5bea6ecfc946f8963149762f9cedad 2023-02-13 16:54:00,767 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/cdbb68a84ad14c75a02c70085d242c88 2023-02-13 16:54:00,773 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/cdbb68a84ad14c75a02c70085d242c88 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/cdbb68a84ad14c75a02c70085d242c88 2023-02-13 16:54:00,779 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/cdbb68a84ad14c75a02c70085d242c88, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:54:00,780 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 834ms, sequenceid=9, compaction requested=false 2023-02-13 16:54:00,780 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:00,780 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:00,780 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:00,783 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,783 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:00,783 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,783 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:00,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:00,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@420b4e25[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:00,785 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:00,785 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,787 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,787 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:00,787 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:00,787 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:00,787 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,787 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:00,789 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:00,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:00,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:00,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:00,792 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:00,792 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:00,792 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:00,792 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,792 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:00,794 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,794 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:00,794 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:00,794 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,796 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:00,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:00,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:00,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:00,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:00,804 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:00,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:00,804 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:00,804 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,805 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,805 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:00,805 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:00,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:00,805 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:00,805 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,805 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,905 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:00,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:00,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:00,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:00,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,914 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:00,914 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:00,914 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:00,914 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:00,915 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,915 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,916 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:00,916 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:00,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:00,917 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:00,917 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:00,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:00,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,918 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,918 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:00,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:00,918 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:00,918 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:00,918 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:00,919 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:00,919 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:00,919 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:00,919 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:00,919 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:01,334 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010b298b1d2bbe4fa990217a01af28554e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010b298b1d2bbe4fa990217a01af28554e 2023-02-13 16:54:01,740 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/7daeda55a2cc46148f7cbafb79041625 2023-02-13 16:54:01,747 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/7daeda55a2cc46148f7cbafb79041625 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/7daeda55a2cc46148f7cbafb79041625 2023-02-13 16:54:01,752 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/7daeda55a2cc46148f7cbafb79041625, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 16:54:01,753 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 834ms, sequenceid=13, compaction requested=true 2023-02-13 16:54:01,753 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:01,753 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:01,753 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:01,753 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:01,754 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,754 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:01,754 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:01,755 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,755 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:01,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:01,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:01,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:01,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:01,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:01,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:01,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6eb8f01e[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:01,757 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:01,757 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,759 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:01,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:01,759 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:01,759 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:01,761 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:01,761 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,761 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:01,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,762 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:01,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:01,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:01,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:01,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:01,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:01,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:01,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:01,765 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:01,765 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:01,765 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:01,765 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,765 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:01,767 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,767 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:01,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,767 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:01,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:01,768 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,769 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:01,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,772 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,772 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:01,772 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:01,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:01,772 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:01,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:01,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:01,772 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:01,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:01,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:01,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,773 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:01,774 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:01,874 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:01,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:01,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:01,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:01,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,882 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:01,882 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:01,883 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:01,883 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:01,884 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,884 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,885 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:01,885 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:01,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:01,886 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,886 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,887 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:01,887 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:01,888 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:01,893 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:01,893 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:01,893 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:01,893 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:01,893 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:01,915 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e192c21b7ce44053ba3cad22c43234e8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205e192c21b7ce44053ba3cad22c43234e8 2023-02-13 16:54:01,923 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:01,924 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/06703f390d78402b8994e142c9c96e50 2023-02-13 16:54:01,930 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/06703f390d78402b8994e142c9c96e50 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/06703f390d78402b8994e142c9c96e50 2023-02-13 16:54:01,937 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/06703f390d78402b8994e142c9c96e50, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 16:54:01,938 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 45ms, sequenceid=17, compaction requested=true 2023-02-13 16:54:01,938 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:01,938 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:01,938 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:01,938 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:01,938 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,938 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:01,938 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:01,941 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,941 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:01,941 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,942 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:01,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35971758[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:01,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:01,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,945 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:01,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:01,946 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,946 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:01,947 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,947 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:01,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:01,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:01,947 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:01,947 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:01,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:01,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:01,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:01,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:01,952 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:01,952 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:01,952 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:01,952 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,952 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:01,954 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,954 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:01,954 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:01,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:01,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:01,955 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:01,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,957 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:01,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:01,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:01,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,962 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:01,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:01,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:01,966 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:01,966 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:01,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:01,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:01,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:01,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:01,966 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:01,967 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:01,967 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:01,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:01,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:02,068 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:02,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:02,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:02,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:02,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,076 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:02,076 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:02,077 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:02,077 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:02,077 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,077 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,079 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,079 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:02,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:02,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:02,079 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,079 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:02,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:02,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:02,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,081 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:02,081 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:02,081 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:02,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:02,082 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:02,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:02,082 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:02,082 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:02,106 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122808f79fbe23044fa08f3e5ee2843c2ad7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122808f79fbe23044fa08f3e5ee2843c2ad7 2023-02-13 16:54:02,521 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/c0e2aa4163274a9d982eb27b6d523cbd 2023-02-13 16:54:02,528 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/c0e2aa4163274a9d982eb27b6d523cbd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/c0e2aa4163274a9d982eb27b6d523cbd 2023-02-13 16:54:02,533 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/c0e2aa4163274a9d982eb27b6d523cbd, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 16:54:02,534 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 452ms, sequenceid=21, compaction requested=true 2023-02-13 16:54:02,534 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:02,534 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:02,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:02,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:02,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:02,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:02,536 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,536 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:02,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:02,536 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,536 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:02,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:02,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:02,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a2c6392[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:02,538 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:02,538 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,539 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,539 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:02,540 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:02,540 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:02,540 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,540 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:02,543 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,543 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:02,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:02,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:02,543 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:02,543 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:02,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:02,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:02,546 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:02,546 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:02,546 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:02,546 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,546 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:02,551 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,551 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:02,551 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,551 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:02,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:02,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:02,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:02,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,552 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:02,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,555 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,555 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:02,555 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:02,555 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:02,555 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:02,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:02,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:02,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:02,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:02,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,556 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,656 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:02,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:02,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:02,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:02,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,664 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:02,664 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:02,665 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:02,665 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:02,665 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,665 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,668 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,668 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:02,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:02,668 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:02,668 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:02,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:02,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:02,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:02,671 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:02,671 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:02,671 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:02,671 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:02,671 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:03,091 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314b97736e66a045b280120cf5e3ee41ae to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512314b97736e66a045b280120cf5e3ee41ae 2023-02-13 16:54:03,497 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/441a2a5312ba4acdb27440bd2b1a83bd 2023-02-13 16:54:03,505 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/441a2a5312ba4acdb27440bd2b1a83bd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/441a2a5312ba4acdb27440bd2b1a83bd 2023-02-13 16:54:03,511 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/441a2a5312ba4acdb27440bd2b1a83bd, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 16:54:03,512 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 841ms, sequenceid=25, compaction requested=true 2023-02-13 16:54:03,512 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:03,512 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:03,512 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:03,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:03,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:03,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:03,514 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,515 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:03,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:03,515 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,515 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:03,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:03,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:03,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:03,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:03,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@524e7694[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:03,517 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:03,517 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,518 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,519 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:03,519 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:03,519 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:03,519 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,519 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:03,520 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:03,520 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,520 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:03,520 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:03,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:03,523 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:03,523 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:03,523 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:03,523 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,523 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:03,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:03,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:03,527 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,528 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:03,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,531 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,531 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,531 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:03,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:03,531 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:03,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:03,531 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:03,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:03,531 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:03,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:03,532 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,532 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,532 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,532 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,532 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:03,532 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:03,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:03,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:03,632 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:03,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:03,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:03,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:03,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,641 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:03,641 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:03,642 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:03,642 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:03,642 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,642 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,644 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,644 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:03,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:03,644 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,644 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,645 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:03,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:03,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:03,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:03,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:03,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:03,646 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:03,646 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:03,647 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:03,665 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c993c6f01d8d45dc863408c694563f21 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101c993c6f01d8d45dc863408c694563f21 2023-02-13 16:54:03,670 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/632036b1e4c7457b9058cba4bb13e0b2 2023-02-13 16:54:03,676 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/632036b1e4c7457b9058cba4bb13e0b2 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/632036b1e4c7457b9058cba4bb13e0b2 2023-02-13 16:54:03,682 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/632036b1e4c7457b9058cba4bb13e0b2, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 16:54:03,683 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 36ms, sequenceid=29, compaction requested=true 2023-02-13 16:54:03,683 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:03,683 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:03,683 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:03,683 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:03,683 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,683 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:03,683 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:03,685 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:03,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:03,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:03,685 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:03,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:03,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:03,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:03,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b5f44c3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:03,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:03,687 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,689 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,689 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:03,689 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:03,689 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:03,689 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,689 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:03,691 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:03,691 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:03,691 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:03,692 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:03,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:03,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:03,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:03,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:03,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:03,695 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:03,695 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:03,695 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:03,695 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,695 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:03,697 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,697 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,697 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:03,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:03,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:03,698 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:03,700 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:03,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:03,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:03,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,703 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:03,704 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,704 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,704 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:03,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:03,703 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:03,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:03,704 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,705 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:03,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:03,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:03,705 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:03,805 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:03,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:03,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:03,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:03,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,817 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:03,817 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:03,817 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:03,817 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:03,818 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,818 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,824 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,824 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:03,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:03,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:03,824 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:03,824 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:03,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:03,828 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:03,828 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:03,828 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:03,828 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:03,828 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:04,252 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e41398813b3b4fab9a9a02eec8c489be to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103e41398813b3b4fab9a9a02eec8c489be 2023-02-13 16:54:04,260 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/8f15a627cb0a4e86a56c53e30b49aae5 2023-02-13 16:54:04,268 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/8f15a627cb0a4e86a56c53e30b49aae5 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/8f15a627cb0a4e86a56c53e30b49aae5 2023-02-13 16:54:04,275 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/8f15a627cb0a4e86a56c53e30b49aae5, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 16:54:04,276 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 448ms, sequenceid=33, compaction requested=true 2023-02-13 16:54:04,277 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:04,277 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:04,279 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:04,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:04,279 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:04,280 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,280 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:04,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:04,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76ffb127[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:04,282 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:04,282 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,285 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,285 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:04,285 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:04,285 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:04,285 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,285 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:04,294 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:04,294 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,294 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:04,294 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:04,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:04,299 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:04,299 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:04,299 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:04,299 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,299 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:04,301 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,301 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:04,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,301 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:04,302 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,303 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,306 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,306 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,306 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:04,306 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:04,306 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:04,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:04,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:04,306 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:04,307 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:04,307 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,308 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,308 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,308 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:04,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:04,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:04,308 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:04,408 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:04,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:04,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:04,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:04,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,418 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:04,418 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:04,418 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:04,418 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:04,419 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,419 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,420 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:04,420 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:04,420 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:04,420 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,421 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,421 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:04,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:04,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:04,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:04,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:04,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:04,422 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:04,422 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:04,423 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:04,840 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907fd852a856b424dc8b630f0477f23574a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907fd852a856b424dc8b630f0477f23574a 2023-02-13 16:54:05,246 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/a98c9786eb634f408a3c2cf327ae2311 2023-02-13 16:54:05,252 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/a98c9786eb634f408a3c2cf327ae2311 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a98c9786eb634f408a3c2cf327ae2311 2023-02-13 16:54:05,257 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a98c9786eb634f408a3c2cf327ae2311, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 16:54:05,258 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 836ms, sequenceid=37, compaction requested=true 2023-02-13 16:54:05,259 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:05,259 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:05,259 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:05,259 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:05,259 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,259 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:05,259 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:05,261 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,261 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:05,261 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,261 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fe66c99[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:05,263 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:05,263 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,265 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:05,265 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,265 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:05,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:05,270 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:05,270 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:05,270 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:05,270 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,270 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:05,272 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,272 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:05,272 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:05,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:05,273 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:05,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,276 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:05,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:05,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:05,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:05,277 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:05,277 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:05,277 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:05,277 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:05,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:05,378 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:05,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:05,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:05,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:05,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,384 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:05,384 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:05,385 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:54:05,385 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:05,385 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,385 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,387 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,387 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:05,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:05,387 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,387 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,388 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,388 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:54:05,388 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:05,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:54:05,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:05,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:05,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:05,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. started... 2023-02-13 16:54:05,390 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d9834553e07b975c30d650299cbec8b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:05,806 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112054eceea3ae1644bfa76c6d97ec3eee29 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112054eceea3ae1644bfa76c6d97ec3eee29 2023-02-13 16:54:05,812 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/3c49dc1d8fbe40b292555014c2b5512c 2023-02-13 16:54:05,817 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/3c49dc1d8fbe40b292555014c2b5512c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/3c49dc1d8fbe40b292555014c2b5512c 2023-02-13 16:54:05,821 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/3c49dc1d8fbe40b292555014c2b5512c, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 16:54:05,822 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d9834553e07b975c30d650299cbec8b5 in 432ms, sequenceid=41, compaction requested=true 2023-02-13 16:54:05,822 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:05,822 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:05,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:54:05,825 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,825 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:05,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,826 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:05,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:54:05,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e3d5b27[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:05,827 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:05,827 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,828 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,829 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:05,829 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:54:05,829 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:05,829 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,829 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:05,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,830 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:54:05,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:05,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:05,830 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:05,831 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:54:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:05,833 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:54:05,833 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:05,833 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:05,833 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,833 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:05,835 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,835 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,835 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:05,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:05,835 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:05,836 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,844 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,844 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,844 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,844 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:05,845 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:05,845 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:05,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:05,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:05,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 16:54:05,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:05,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:54:05,845 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:05,846 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:05,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:05,846 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,846 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:05,846 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,846 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:05,947 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:05,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:05,949 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-13 16:54:05,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:05,951 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:05,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:05,952 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307845952 2023-02-13 16:54:05,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:54:05,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-13 16:54:05,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:05,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x5baeff93 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:05,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d08962b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:05,959 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:06,798 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103510331009fd142598d1301d9ca930b40 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103510331009fd142598d1301d9ca930b40 2023-02-13 16:54:06,799 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:06,812 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:06,813 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36290, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:06,820 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:06,821 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35928, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:06,831 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0f2df60065594450b10c473d85a7bfc2 first=Optional[r5] last=Optional[r6] 2023-02-13 16:54:06,831 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0f2df60065594450b10c473d85a7bfc2}] 2023-02-13 16:54:06,835 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0f2df60065594450b10c473d85a7bfc2 for inclusion in d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:54:06,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-13 16:54:06,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:06,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:06,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0f2df60065594450b10c473d85a7bfc2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__796s2qha733jpcgi2ncv3iegvhapc38umd7hj711ahoia1sgf022lgakfbevu8op/family1/0f2df60065594450b10c473d85a7bfc2 2023-02-13 16:54:06,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__796s2qha733jpcgi2ncv3iegvhapc38umd7hj711ahoia1sgf022lgakfbevu8op/family1/0f2df60065594450b10c473d85a7bfc2 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_ 2023-02-13 16:54:06,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__796s2qha733jpcgi2ncv3iegvhapc38umd7hj711ahoia1sgf022lgakfbevu8op/family1/0f2df60065594450b10c473d85a7bfc2 into d9834553e07b975c30d650299cbec8b5/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_ - updating store file list. 2023-02-13 16:54:06,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_ into d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:54:06,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__796s2qha733jpcgi2ncv3iegvhapc38umd7hj711ahoia1sgf022lgakfbevu8op/family1/0f2df60065594450b10c473d85a7bfc2 into d9834553e07b975c30d650299cbec8b5/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_) 2023-02-13 16:54:06,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__796s2qha733jpcgi2ncv3iegvhapc38umd7hj711ahoia1sgf022lgakfbevu8op/family1/0f2df60065594450b10c473d85a7bfc2 2023-02-13 16:54:06,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:54:06,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:54:06,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): d9834553e07b975c30d650299cbec8b5/family1 is initiating minor compaction 2023-02-13 16:54:06,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@151d6e93; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:06,852 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d9834553e07b975c30d650299cbec8b5/family1 in testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:06,852 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/cdbb68a84ad14c75a02c70085d242c88, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/7daeda55a2cc46148f7cbafb79041625, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/06703f390d78402b8994e142c9c96e50, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/c0e2aa4163274a9d982eb27b6d523cbd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/441a2a5312ba4acdb27440bd2b1a83bd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/632036b1e4c7457b9058cba4bb13e0b2, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/8f15a627cb0a4e86a56c53e30b49aae5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a98c9786eb634f408a3c2cf327ae2311, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/3c49dc1d8fbe40b292555014c2b5512c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp, totalSize=49.7 K 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting cdbb68a84ad14c75a02c70085d242c88, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 7daeda55a2cc46148f7cbafb79041625, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 06703f390d78402b8994e142c9c96e50, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting c0e2aa4163274a9d982eb27b6d523cbd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 441a2a5312ba4acdb27440bd2b1a83bd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 632036b1e4c7457b9058cba4bb13e0b2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:54:06,852 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 8f15a627cb0a4e86a56c53e30b49aae5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:54:06,853 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting a98c9786eb634f408a3c2cf327ae2311, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:54:06,853 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 3c49dc1d8fbe40b292555014c2b5512c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:54:06,853 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:54:06,854 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:06,855 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:06,857 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101c993c6f01d8d45dc863408c694563f21 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101c993c6f01d8d45dc863408c694563f21 2023-02-13 16:54:06,859 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103e41398813b3b4fab9a9a02eec8c489be to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103e41398813b3b4fab9a9a02eec8c489be 2023-02-13 16:54:06,859 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-13 16:54:06,859 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-13 16:54:06,859 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:06,859 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:06,903 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): d9834553e07b975c30d650299cbec8b5#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:07,331 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/5c17527ded3241e386daea2d1459864e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5c17527ded3241e386daea2d1459864e 2023-02-13 16:54:07,337 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d9834553e07b975c30d650299cbec8b5/family1 of d9834553e07b975c30d650299cbec8b5 into 5c17527ded3241e386daea2d1459864e(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:07,338 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:07,338 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676307246851; duration=0sec 2023-02-13 16:54:07,338 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:07,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/cdbb68a84ad14c75a02c70085d242c88, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/7daeda55a2cc46148f7cbafb79041625, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/06703f390d78402b8994e142c9c96e50, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/c0e2aa4163274a9d982eb27b6d523cbd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/441a2a5312ba4acdb27440bd2b1a83bd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/632036b1e4c7457b9058cba4bb13e0b2, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/8f15a627cb0a4e86a56c53e30b49aae5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a98c9786eb634f408a3c2cf327ae2311, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/3c49dc1d8fbe40b292555014c2b5512c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_] to archive 2023-02-13 16:54:07,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:07,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/cdbb68a84ad14c75a02c70085d242c88 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/cdbb68a84ad14c75a02c70085d242c88 2023-02-13 16:54:07,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/7daeda55a2cc46148f7cbafb79041625 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/7daeda55a2cc46148f7cbafb79041625 2023-02-13 16:54:07,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/06703f390d78402b8994e142c9c96e50 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/06703f390d78402b8994e142c9c96e50 2023-02-13 16:54:07,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/c0e2aa4163274a9d982eb27b6d523cbd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/c0e2aa4163274a9d982eb27b6d523cbd 2023-02-13 16:54:07,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/441a2a5312ba4acdb27440bd2b1a83bd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/441a2a5312ba4acdb27440bd2b1a83bd 2023-02-13 16:54:07,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/632036b1e4c7457b9058cba4bb13e0b2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/632036b1e4c7457b9058cba4bb13e0b2 2023-02-13 16:54:07,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/8f15a627cb0a4e86a56c53e30b49aae5 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/8f15a627cb0a4e86a56c53e30b49aae5 2023-02-13 16:54:07,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a98c9786eb634f408a3c2cf327ae2311 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a98c9786eb634f408a3c2cf327ae2311 2023-02-13 16:54:07,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/3c49dc1d8fbe40b292555014c2b5512c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/3c49dc1d8fbe40b292555014c2b5512c 2023-02-13 16:54:07,448 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5caeeefcc5774b2b87fb5c8f70313bcc_SeqId_44_ 2023-02-13 16:54:07,728 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316a144c38a3a44552ab9175677894bfc2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512316a144c38a3a44552ab9175677894bfc2 2023-02-13 16:54:07,729 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:07,754 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/24ac921c26e3487fb2471ff28f4ac9c1 first=Optional[r1] last=Optional[r4] 2023-02-13 16:54:07,755 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/24ac921c26e3487fb2471ff28f4ac9c1}] 2023-02-13 16:54:07,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/24ac921c26e3487fb2471ff28f4ac9c1 for inclusion in d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:54:07,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-13 16:54:07,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:07,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:07,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/24ac921c26e3487fb2471ff28f4ac9c1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u8bidjjbfsavt7q8nq35dbd5qhnq7f5tc1a90sjgpesg4drf66qe1l6f91g3raeg/family1/24ac921c26e3487fb2471ff28f4ac9c1 2023-02-13 16:54:07,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u8bidjjbfsavt7q8nq35dbd5qhnq7f5tc1a90sjgpesg4drf66qe1l6f91g3raeg/family1/24ac921c26e3487fb2471ff28f4ac9c1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_ 2023-02-13 16:54:07,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u8bidjjbfsavt7q8nq35dbd5qhnq7f5tc1a90sjgpesg4drf66qe1l6f91g3raeg/family1/24ac921c26e3487fb2471ff28f4ac9c1 into d9834553e07b975c30d650299cbec8b5/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_ - updating store file list. 2023-02-13 16:54:07,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_ into d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:54:07,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u8bidjjbfsavt7q8nq35dbd5qhnq7f5tc1a90sjgpesg4drf66qe1l6f91g3raeg/family1/24ac921c26e3487fb2471ff28f4ac9c1 into d9834553e07b975c30d650299cbec8b5/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_) 2023-02-13 16:54:07,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u8bidjjbfsavt7q8nq35dbd5qhnq7f5tc1a90sjgpesg4drf66qe1l6f91g3raeg/family1/24ac921c26e3487fb2471ff28f4ac9c1 2023-02-13 16:54:07,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:54:07,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:54:07,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): d9834553e07b975c30d650299cbec8b5/family1 is initiating minor compaction (all files) 2023-02-13 16:54:07,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3399169b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:07,774 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d9834553e07b975c30d650299cbec8b5/family1 in testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:54:07,774 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2af4eefa9f54610b991c0c0afaf1c98, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5c17527ded3241e386daea2d1459864e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp, totalSize=178.1 K 2023-02-13 16:54:07,775 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting e2af4eefa9f54610b991c0c0afaf1c98, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842816 2023-02-13 16:54:07,775 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5c17527ded3241e386daea2d1459864e, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242816 2023-02-13 16:54:07,776 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting a8aeb01432954acfae25f5876fc62060_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242816 2023-02-13 16:54:07,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:07,777 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:07,778 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010b298b1d2bbe4fa990217a01af28554e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512010b298b1d2bbe4fa990217a01af28554e 2023-02-13 16:54:07,781 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205e192c21b7ce44053ba3cad22c43234e8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205e192c21b7ce44053ba3cad22c43234e8 2023-02-13 16:54:07,783 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122808f79fbe23044fa08f3e5ee2843c2ad7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122808f79fbe23044fa08f3e5ee2843c2ad7 2023-02-13 16:54:07,784 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512314b97736e66a045b280120cf5e3ee41ae to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512314b97736e66a045b280120cf5e3ee41ae 2023-02-13 16:54:07,784 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-13 16:54:07,784 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-13 16:54:07,785 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:07,785 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:07,795 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): d9834553e07b975c30d650299cbec8b5#family1#compaction#23 average throughput is 19.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:07,836 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/.tmp/family1/e2d23cf03fd4426b8eb577fa0e48b40c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2d23cf03fd4426b8eb577fa0e48b40c 2023-02-13 16:54:07,843 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309349604195104d6aac2fab914e34712e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309349604195104d6aac2fab914e34712e 2023-02-13 16:54:07,844 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d9834553e07b975c30d650299cbec8b5/family1 of d9834553e07b975c30d650299cbec8b5 into e2d23cf03fd4426b8eb577fa0e48b40c(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:07,844 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:07,844 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676307247774; duration=0sec 2023-02-13 16:54:07,844 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:07,844 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:07,881 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/60ad4907eb70486a8a07f9b8106c522b first=Optional[r00] last=Optional[r8] 2023-02-13 16:54:07,882 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/60ad4907eb70486a8a07f9b8106c522b}] 2023-02-13 16:54:07,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/60ad4907eb70486a8a07f9b8106c522b for inclusion in d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:54:07,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-13 16:54:07,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:07,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for d9834553e07b975c30d650299cbec8b5: 2023-02-13 16:54:07,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/60ad4907eb70486a8a07f9b8106c522b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7antini2t9d92dbjrkfjkoafiqqgs4anhs7opeb53gtutorcuohemt14busqj1tr/family1/60ad4907eb70486a8a07f9b8106c522b 2023-02-13 16:54:07,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7antini2t9d92dbjrkfjkoafiqqgs4anhs7opeb53gtutorcuohemt14busqj1tr/family1/60ad4907eb70486a8a07f9b8106c522b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/031e5135bfb1427bbe5b84dbfa7e5bd6_SeqId_50_ 2023-02-13 16:54:07,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7antini2t9d92dbjrkfjkoafiqqgs4anhs7opeb53gtutorcuohemt14busqj1tr/family1/60ad4907eb70486a8a07f9b8106c522b into d9834553e07b975c30d650299cbec8b5/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/031e5135bfb1427bbe5b84dbfa7e5bd6_SeqId_50_ - updating store file list. 2023-02-13 16:54:07,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/031e5135bfb1427bbe5b84dbfa7e5bd6_SeqId_50_ into d9834553e07b975c30d650299cbec8b5/family1 2023-02-13 16:54:07,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7antini2t9d92dbjrkfjkoafiqqgs4anhs7opeb53gtutorcuohemt14busqj1tr/family1/60ad4907eb70486a8a07f9b8106c522b into d9834553e07b975c30d650299cbec8b5/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/031e5135bfb1427bbe5b84dbfa7e5bd6_SeqId_50_) 2023-02-13 16:54:07,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithMonthlyPolicy__7antini2t9d92dbjrkfjkoafiqqgs4anhs7opeb53gtutorcuohemt14busqj1tr/family1/60ad4907eb70486a8a07f9b8106c522b 2023-02-13 16:54:07,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:07,901 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:07,902 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112054eceea3ae1644bfa76c6d97ec3eee29 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112054eceea3ae1644bfa76c6d97ec3eee29 2023-02-13 16:54:07,903 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511282f5bea6ecfc946f8963149762f9cedad to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511282f5bea6ecfc946f8963149762f9cedad 2023-02-13 16:54:07,904 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d66f0e34923c430a8640756ab6df6ef0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d66f0e34923c430a8640756ab6df6ef0 2023-02-13 16:54:07,905 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-13 16:54:07,905 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 16:54:07,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:07,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:07,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:54:07,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:54:07,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:07,907 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:07,909 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9580 sec 2023-02-13 16:54:07,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2af4eefa9f54610b991c0c0afaf1c98, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5c17527ded3241e386daea2d1459864e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_] to archive 2023-02-13 16:54:07,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:07,930 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2af4eefa9f54610b991c0c0afaf1c98 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/e2af4eefa9f54610b991c0c0afaf1c98 2023-02-13 16:54:07,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5c17527ded3241e386daea2d1459864e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/5c17527ded3241e386daea2d1459864e 2023-02-13 16:54:07,933 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1/a8aeb01432954acfae25f5876fc62060_SeqId_47_ 2023-02-13 16:54:07,943 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 17 more 2023-02-13 16:54:07,977 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:54:07,977 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4fbb4281 to 127.0.0.1:64619 2023-02-13 16:54:07,978 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:54:07,979 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:54:07,979 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ddb30aa to 127.0.0.1:64619 2023-02-13 16:54:07,979 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:54:07,979 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:54:07,980 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5baeff93 to 127.0.0.1:64619 2023-02-13 16:54:07,980 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:54:08,030 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249 (was 248) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x5baeff93-SendThread(127.0.0.1:64619) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1261) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42274 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42370 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42282 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42214 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42300 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_889282984_15 at /127.0.0.1:42308 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42420 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1638776d-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:34756 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42404 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42258 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42332 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x5baeff93-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_889282984_15 at /127.0.0.1:42236 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42402 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42482 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42318 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42386 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1227896652_15 at /127.0.0.1:42484 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64619@0x5baeff93 java.lang.Object.wait(Native Method) java.lang.Thread.join(Thread.java:1257) java.lang.Thread.join(Thread.java:1331) org.apache.zookeeper.ClientCnxn.disconnect(ClientCnxn.java:1455) org.apache.zookeeper.ClientCnxn.close(ClientCnxn.java:1485) org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:1415) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.closeZk(ReadOnlyZKClient.java:304) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:352) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1479189715.run(Unknown Source) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=599 (was 591) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=336 (was 387), ProcessCount=168 (was 168), AvailableMemoryMB=1999 (was 1339) - AvailableMemoryMB LEAK? - 2023-02-13 16:54:08,042 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=249, OpenFileDescriptor=599, MaxFileDescriptor=60000, SystemLoadAverage=336, ProcessCount=168, AvailableMemoryMB=1998 2023-02-13 16:54:08,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 16:54:08,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:08,048 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:08,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-13 16:54:08,049 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:08,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 16:54:08,051 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,051 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7 empty. 2023-02-13 16:54:08,052 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,052 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-13 16:54:08,061 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:08,063 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3f0364c8ae730af4bb790813ec085cf7, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:08,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 16:54:08,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 16:54:08,471 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:08,471 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 3f0364c8ae730af4bb790813ec085cf7, disabling compactions & flushes 2023-02-13 16:54:08,471 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,471 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,471 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. after waiting 0 ms 2023-02-13 16:54:08,471 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,471 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,472 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 3f0364c8ae730af4bb790813ec085cf7: Waiting for close lock at 1676307248471Disabling compacts and flushes for region at 1676307248471Disabling writes for close at 1676307248471Writing region close event to WAL at 1676307248471Closed at 1676307248471 2023-02-13 16:54:08,474 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:08,475 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307248475"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307248475"}]},"ts":"1676307248475"} 2023-02-13 16:54:08,477 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:08,478 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:08,478 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307248478"}]},"ts":"1676307248478"} 2023-02-13 16:54:08,479 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-13 16:54:08,490 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=3f0364c8ae730af4bb790813ec085cf7, ASSIGN}] 2023-02-13 16:54:08,492 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=3f0364c8ae730af4bb790813ec085cf7, ASSIGN 2023-02-13 16:54:08,493 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=3f0364c8ae730af4bb790813ec085cf7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:08,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=3f0364c8ae730af4bb790813ec085cf7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:08,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307248644"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307248644"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307248644"}]},"ts":"1676307248644"} 2023-02-13 16:54:08,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 3f0364c8ae730af4bb790813ec085cf7, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:08,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 16:54:08,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3f0364c8ae730af4bb790813ec085cf7, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:08,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:08,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,803 INFO [StoreOpener-3f0364c8ae730af4bb790813ec085cf7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,804 DEBUG [StoreOpener-3f0364c8ae730af4bb790813ec085cf7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1 2023-02-13 16:54:08,804 DEBUG [StoreOpener-3f0364c8ae730af4bb790813ec085cf7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1 2023-02-13 16:54:08,804 INFO [StoreOpener-3f0364c8ae730af4bb790813ec085cf7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3f0364c8ae730af4bb790813ec085cf7 columnFamilyName family1 2023-02-13 16:54:08,806 INFO [StoreOpener-3f0364c8ae730af4bb790813ec085cf7-1] regionserver.HStore(310): Store=3f0364c8ae730af4bb790813ec085cf7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:08,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:54:08,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:08,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3f0364c8ae730af4bb790813ec085cf7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69317432, jitterRate=0.03291022777557373}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:08,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3f0364c8ae730af4bb790813ec085cf7: Running coprocessor pre-open hook at 1676307248802Writing region info on filesystem at 1676307248802Initializing all the Stores at 1676307248802Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676307248802Cleaning up temporary data from old regions at 1676307248807 (+5 ms)Cleaning up detritus from prior splits at 1676307248808 (+1 ms)Running coprocessor post-open hooks at 1676307248812 (+4 ms)Region opened successfully at 1676307248812 2023-02-13 16:54:08,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7., pid=25, masterSystemTime=1676307248798 2023-02-13 16:54:08,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:08,815 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=3f0364c8ae730af4bb790813ec085cf7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:08,815 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307248815"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307248815"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307248815"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307248815"}]},"ts":"1676307248815"} 2023-02-13 16:54:08,819 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-13 16:54:08,819 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 3f0364c8ae730af4bb790813ec085cf7, server=jenkins-hbase4.apache.org,33907,1676307219708 in 171 msec 2023-02-13 16:54:08,821 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-13 16:54:08,821 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=3f0364c8ae730af4bb790813ec085cf7, ASSIGN in 329 msec 2023-02-13 16:54:08,822 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:08,822 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307248822"}]},"ts":"1676307248822"} 2023-02-13 16:54:08,823 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-13 16:54:08,827 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:08,828 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 782 msec 2023-02-13 16:54:09,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 16:54:09,153 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-13 16:54:09,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:09,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,184 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:09,184 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:09,184 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:09,184 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:09,184 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,184 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,186 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:09,186 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:09,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:09,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:09,186 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:09,186 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:09,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:09,187 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:09,187 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:09,187 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:09,187 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:09,187 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:09,187 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:09,187 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:09,188 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:54:09,608 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b1f22169e2084aab99d6816d4546e870 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130b1f22169e2084aab99d6816d4546e870 2023-02-13 16:54:10,012 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/26fce9fd292f4e6aba2c1fedbaf76ddf 2023-02-13 16:54:10,018 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/26fce9fd292f4e6aba2c1fedbaf76ddf as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/26fce9fd292f4e6aba2c1fedbaf76ddf 2023-02-13 16:54:10,022 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/26fce9fd292f4e6aba2c1fedbaf76ddf, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-13 16:54:10,023 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 835ms, sequenceid=5, compaction requested=false 2023-02-13 16:54:10,023 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:10,024 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:10,024 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:10,026 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,026 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:10,027 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,027 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:10,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:10,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:10,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:10,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fc5617f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:10,028 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:10,028 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,030 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:10,030 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:10,030 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,030 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:10,030 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:10,031 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,031 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:10,031 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:10,031 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:10,033 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:10,033 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:10,033 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:10,033 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,033 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:10,036 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,036 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,036 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:10,037 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:10,038 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,040 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:10,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:10,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:10,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:10,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:10,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:10,141 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:10,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:10,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:10,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:10,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,148 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:10,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:10,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:10,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:10,149 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,149 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,150 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,150 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:10,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:10,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:10,150 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,150 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:10,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,152 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:10,152 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:10,152 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:10,152 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:10,152 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:10,152 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:10,152 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:10,152 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:54:10,568 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112872a7fe1be311435bbcaf265bd76c8a9f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112872a7fe1be311435bbcaf265bd76c8a9f 2023-02-13 16:54:10,974 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/8a71400a2d5c462f85e199c28ced9e1f 2023-02-13 16:54:10,979 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/8a71400a2d5c462f85e199c28ced9e1f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/8a71400a2d5c462f85e199c28ced9e1f 2023-02-13 16:54:10,984 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/8a71400a2d5c462f85e199c28ced9e1f, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:54:10,985 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 833ms, sequenceid=9, compaction requested=false 2023-02-13 16:54:10,985 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:10,985 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:10,985 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:10,985 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:10,985 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,985 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:10,985 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:10,987 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,987 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:10,987 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,987 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74b89129[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:10,989 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:10,989 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,991 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,991 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:10,991 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:10,991 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,991 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:10,991 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:10,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:10,992 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,993 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:10,993 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:10,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:10,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:10,995 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:10,995 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:10,995 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:10,995 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,995 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:10,997 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,997 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:10,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,997 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:10,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:10,997 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:10,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,998 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:11,001 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:11,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:11,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:11,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:11,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:11,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:11,001 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:11,102 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:11,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:11,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:11,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:11,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,109 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:11,110 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:11,110 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:11,110 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:11,110 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,110 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,112 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,112 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:11,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:11,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:11,113 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,113 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:11,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:11,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:11,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,114 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:11,114 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:11,114 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:11,114 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:11,114 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:11,114 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:11,114 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:11,115 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:11,534 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120177eec562c9004558ac208ca7cb1dfd8e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120177eec562c9004558ac208ca7cb1dfd8e 2023-02-13 16:54:11,539 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/bf4f5c8cbadc4656aa45e980d512e1b0 2023-02-13 16:54:11,545 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/bf4f5c8cbadc4656aa45e980d512e1b0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/bf4f5c8cbadc4656aa45e980d512e1b0 2023-02-13 16:54:11,549 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/bf4f5c8cbadc4656aa45e980d512e1b0, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 16:54:11,550 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 435ms, sequenceid=13, compaction requested=true 2023-02-13 16:54:11,550 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:11,550 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:11,551 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:11,551 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:11,551 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,551 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:11,551 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:11,552 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,552 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:11,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:11,552 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:11,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:11,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18d6b983[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:11,554 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:11,554 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,555 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,555 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:11,555 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:11,555 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:11,555 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,555 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:11,557 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,557 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:11,557 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:11,557 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:11,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:11,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:11,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:11,559 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:11,559 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:11,559 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:11,559 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,559 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:11,564 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,564 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:11,564 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,565 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:11,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:11,566 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:11,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:11,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:11,568 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:11,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:11,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:11,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:11,568 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,569 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:11,569 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:11,569 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:11,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:11,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:11,669 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:11,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:11,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:11,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:11,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,676 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:11,676 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:11,676 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:11,676 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:11,677 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,677 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,678 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:11,678 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:11,679 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:11,679 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,679 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,679 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,679 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:11,680 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:11,680 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:11,680 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:12,093 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205dc18c1d2487c4aa0b444a0c20f358895 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205dc18c1d2487c4aa0b444a0c20f358895 2023-02-13 16:54:12,499 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/82533a170cf74170942c2961ecdaa7ec 2023-02-13 16:54:12,504 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/82533a170cf74170942c2961ecdaa7ec as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/82533a170cf74170942c2961ecdaa7ec 2023-02-13 16:54:12,509 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/82533a170cf74170942c2961ecdaa7ec, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 16:54:12,510 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 830ms, sequenceid=17, compaction requested=true 2023-02-13 16:54:12,510 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:12,510 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:12,510 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:12,510 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:12,510 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,510 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:12,510 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:12,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,513 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:12,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:12,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,513 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:12,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:12,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:12,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:12,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:12,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c262ddd[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:12,515 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:12,515 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,516 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:12,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:12,516 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,516 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:12,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:12,518 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,518 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:12,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:12,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:12,518 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:12,518 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:12,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:12,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:12,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:12,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:12,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:12,520 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:12,520 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:12,520 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:12,520 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,520 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:12,521 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,521 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:12,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:12,521 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:12,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:12,522 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:12,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:12,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:12,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,523 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:12,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:12,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:12,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,526 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:12,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:12,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:12,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,527 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:12,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:12,627 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:12,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:12,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:12,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:12,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,633 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:12,633 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:12,633 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:12,633 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:12,634 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,634 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,635 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:12,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:12,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:12,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:12,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:12,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:12,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:12,637 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:12,637 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:12,637 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:13,051 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228394aab5bf817472dbe0331bb4f6cfd91 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228394aab5bf817472dbe0331bb4f6cfd91 2023-02-13 16:54:13,457 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/ebe7be09c8fc4e59a173922c7af213f7 2023-02-13 16:54:13,462 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/ebe7be09c8fc4e59a173922c7af213f7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ebe7be09c8fc4e59a173922c7af213f7 2023-02-13 16:54:13,467 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ebe7be09c8fc4e59a173922c7af213f7, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 16:54:13,468 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 831ms, sequenceid=21, compaction requested=true 2023-02-13 16:54:13,468 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:13,468 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:13,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:13,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:13,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:13,469 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:13,470 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:13,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:13,470 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:13,471 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,471 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:13,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:13,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:13,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:13,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58bc6bbf[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:13,472 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:13,472 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,473 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,474 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:13,474 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:13,474 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:13,474 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,474 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:13,477 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:13,477 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,477 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:13,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:13,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:13,477 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:13,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:13,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:13,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:13,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:13,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:13,479 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:13,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:13,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:13,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,479 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:13,481 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,481 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:13,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:13,481 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:13,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,481 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:13,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:13,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:13,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:13,482 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:13,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,484 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,484 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,484 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:13,484 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:13,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:13,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:13,484 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:13,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:13,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:13,484 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:13,485 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,485 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,485 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:13,485 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:13,485 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:13,485 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:13,585 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:13,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:13,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:13,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:13,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,592 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:13,592 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:13,592 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:13,592 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:13,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,594 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:13,594 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:13,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:13,594 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:13,595 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:13,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:13,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:13,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:13,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:13,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:13,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:13,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:13,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:13,596 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:13,596 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:13,596 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:14,012 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312814d6ea59ab416e874feafb88e2c110 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512312814d6ea59ab416e874feafb88e2c110 2023-02-13 16:54:14,330 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:14,417 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/9da48c045e7b426d87c79342302938f4 2023-02-13 16:54:14,423 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/9da48c045e7b426d87c79342302938f4 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/9da48c045e7b426d87c79342302938f4 2023-02-13 16:54:14,427 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/9da48c045e7b426d87c79342302938f4, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 16:54:14,428 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 832ms, sequenceid=25, compaction requested=true 2023-02-13 16:54:14,429 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:14,429 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:14,429 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:14,429 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:14,429 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,429 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:14,429 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:14,432 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:14,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:14,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:14,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@683d9858[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:14,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:14,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,435 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,435 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:14,435 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:14,435 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:14,435 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,436 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:14,440 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:14,440 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:14,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,440 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:14,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:14,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:14,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:14,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:14,443 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:14,443 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:14,443 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:14,443 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,443 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:14,445 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,445 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,445 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:14,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:14,445 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:14,446 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:14,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,448 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,448 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,448 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:14,448 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:14,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:14,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:14,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:14,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:14,448 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,550 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:14,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:14,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:14,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:14,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,557 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:14,557 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:14,557 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:14,557 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:14,557 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,557 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,558 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,558 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:14,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:14,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:14,559 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:14,559 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:14,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,559 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:14,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,559 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,560 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:14,560 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:14,560 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:14,560 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:14,560 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:14,560 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:14,560 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:14,560 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:14,561 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:14,561 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:14,974 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101eb5a7f72f1f5406c9652fc7086878ae0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101eb5a7f72f1f5406c9652fc7086878ae0 2023-02-13 16:54:15,379 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/765535e91b8b45639a2a4e447766cfcd 2023-02-13 16:54:15,385 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/765535e91b8b45639a2a4e447766cfcd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/765535e91b8b45639a2a4e447766cfcd 2023-02-13 16:54:15,389 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/765535e91b8b45639a2a4e447766cfcd, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 16:54:15,390 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 829ms, sequenceid=29, compaction requested=true 2023-02-13 16:54:15,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:15,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:15,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:15,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:15,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:15,390 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:15,392 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,392 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:15,392 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,392 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:15,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:15,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@450a2e7d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:15,393 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:15,393 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,394 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,395 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:15,395 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:15,395 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,395 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:15,395 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:15,398 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:15,397 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,398 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:15,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:15,400 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:15,400 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:15,400 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:15,400 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,400 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:15,402 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,402 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:15,402 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,402 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:15,403 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,405 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:15,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:15,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,406 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:15,406 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:15,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:15,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:15,406 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:15,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:15,406 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,406 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:15,406 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:15,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:15,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:15,506 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:15,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:15,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:15,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:15,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,513 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:15,513 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:15,513 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:15,513 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:15,513 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,513 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,515 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:15,515 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:15,515 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:15,515 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:15,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:15,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:15,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:15,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:15,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:15,516 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:15,516 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:15,516 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:15,516 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:15,929 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034bc069c6d16e45c7aa0ced8a5027b308 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601034bc069c6d16e45c7aa0ced8a5027b308 2023-02-13 16:54:16,334 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/98ba8ee31fe04bebadaafaab2ee695fa 2023-02-13 16:54:16,340 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/98ba8ee31fe04bebadaafaab2ee695fa as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/98ba8ee31fe04bebadaafaab2ee695fa 2023-02-13 16:54:16,344 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/98ba8ee31fe04bebadaafaab2ee695fa, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 16:54:16,345 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 829ms, sequenceid=33, compaction requested=true 2023-02-13 16:54:16,345 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:16,345 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:16,345 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:16,345 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:16,345 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,345 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:16,345 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,348 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:16,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:16,349 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,349 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:16,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:16,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:16,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78ffabc4[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:16,350 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:16,350 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,351 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,351 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:16,351 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:16,351 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:16,351 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,351 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:16,353 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,353 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:16,353 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,353 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:16,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:16,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:16,355 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:16,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:16,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:16,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,355 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:16,356 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,356 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,356 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:16,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:16,357 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:16,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,358 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:16,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,361 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,361 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,361 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,361 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,361 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:16,361 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:16,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:16,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:16,361 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:16,362 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:16,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:16,362 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:16,362 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,362 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:16,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:16,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:16,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:16,362 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,462 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:16,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:16,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:16,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:16,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,469 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:16,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:16,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:16,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:16,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,470 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:16,470 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:16,471 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:16,471 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:16,472 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:16,473 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:16,473 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:16,886 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509073dfb4a40d72d4668a849964968c5ab18 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509073dfb4a40d72d4668a849964968c5ab18 2023-02-13 16:54:17,292 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/b7b46058767e47f4927aaec30f2b39a0 2023-02-13 16:54:17,298 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/b7b46058767e47f4927aaec30f2b39a0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/b7b46058767e47f4927aaec30f2b39a0 2023-02-13 16:54:17,304 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/b7b46058767e47f4927aaec30f2b39a0, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 16:54:17,305 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 832ms, sequenceid=37, compaction requested=true 2023-02-13 16:54:17,305 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:17,305 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:17,305 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:17,305 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:17,305 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,305 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:17,305 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:17,307 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,307 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:17,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:17,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:17,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4325c6f7[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:17,310 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:17,310 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,315 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,315 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:17,315 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:17,315 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:17,315 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,315 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:17,316 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:17,317 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,317 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:17,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,317 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:17,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:17,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:17,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:17,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:17,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:17,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:17,319 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:17,319 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:17,319 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:17,319 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,319 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:17,321 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,321 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:17,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:17,321 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:17,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:17,322 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:17,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:17,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:17,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,323 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:17,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:17,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:17,327 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,327 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:17,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:17,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:17,327 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:17,327 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:17,327 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:17,327 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:17,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:17,327 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:17,328 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:17,428 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:17,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:17,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:17,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:17,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,435 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:17,435 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:17,435 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:17,435 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:17,435 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,435 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,436 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,436 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:17,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:17,437 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:17,437 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:17,438 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:17,438 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. started... 2023-02-13 16:54:17,439 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f0364c8ae730af4bb790813ec085cf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:17,852 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120abd534fd80954788ae49320385135e91 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120abd534fd80954788ae49320385135e91 2023-02-13 16:54:18,260 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/ec4f7b2916e049e5808d5853d5549355 2023-02-13 16:54:18,267 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/ec4f7b2916e049e5808d5853d5549355 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ec4f7b2916e049e5808d5853d5549355 2023-02-13 16:54:18,272 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ec4f7b2916e049e5808d5853d5549355, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 16:54:18,273 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3f0364c8ae730af4bb790813ec085cf7 in 834ms, sequenceid=41, compaction requested=true 2023-02-13 16:54:18,273 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:18,273 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:18,273 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:18,273 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:18,273 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,273 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:18,273 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:18,275 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,275 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:18,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:18,276 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,276 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:18,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:18,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:18,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:18,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:18,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@475e7993[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:18,277 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:18,277 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,279 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:18,279 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,279 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:18,281 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,281 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:18,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:18,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:18,281 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:18,282 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:18,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:18,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:18,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:18,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:18,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:18,284 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:18,284 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:18,284 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:18,284 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,284 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:18,286 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,286 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,286 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:18,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:18,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:18,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:18,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:18,287 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:18,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,288 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:18,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,290 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,290 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:18,290 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 16:54:18,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:18,290 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:18,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:18,290 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:18,290 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:18,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:18,291 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:18,291 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:18,290 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:18,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:18,291 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:18,291 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,291 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,391 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:18,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:18,393 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-13 16:54:18,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:18,396 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:18,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:18,397 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307858397 2023-02-13 16:54:18,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 16:54:18,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-13 16:54:18,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:18,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x04df23ee to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:18,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5dbf4490, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:18,404 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,879 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010324b1a1dd8e9440508e0196bd71b29407 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010324b1a1dd8e9440508e0196bd71b29407 2023-02-13 16:54:18,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:18,893 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:18,894 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55802, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:18,901 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:18,903 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49396, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:18,913 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a28c68d227e54243a8319f66ec9d4d07 first=Optional[r3] last=Optional[r6] 2023-02-13 16:54:18,913 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a28c68d227e54243a8319f66ec9d4d07}] 2023-02-13 16:54:18,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a28c68d227e54243a8319f66ec9d4d07 for inclusion in 3f0364c8ae730af4bb790813ec085cf7/family1 2023-02-13 16:54:18,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 16:54:18,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:18,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:18,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a28c68d227e54243a8319f66ec9d4d07 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bdpba9eeaso9hn6h015i5aihql1c9iifj6706vn4bfirudrc0jto8pp1o8ecfp8t/family1/a28c68d227e54243a8319f66ec9d4d07 2023-02-13 16:54:18,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bdpba9eeaso9hn6h015i5aihql1c9iifj6706vn4bfirudrc0jto8pp1o8ecfp8t/family1/a28c68d227e54243a8319f66ec9d4d07 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_ 2023-02-13 16:54:18,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bdpba9eeaso9hn6h015i5aihql1c9iifj6706vn4bfirudrc0jto8pp1o8ecfp8t/family1/a28c68d227e54243a8319f66ec9d4d07 into 3f0364c8ae730af4bb790813ec085cf7/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_ - updating store file list. 2023-02-13 16:54:18,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_ into 3f0364c8ae730af4bb790813ec085cf7/family1 2023-02-13 16:54:18,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bdpba9eeaso9hn6h015i5aihql1c9iifj6706vn4bfirudrc0jto8pp1o8ecfp8t/family1/a28c68d227e54243a8319f66ec9d4d07 into 3f0364c8ae730af4bb790813ec085cf7/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_) 2023-02-13 16:54:18,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__bdpba9eeaso9hn6h015i5aihql1c9iifj6706vn4bfirudrc0jto8pp1o8ecfp8t/family1/a28c68d227e54243a8319f66ec9d4d07 2023-02-13 16:54:18,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:54:18,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:54:18,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): 3f0364c8ae730af4bb790813ec085cf7/family1 is initiating minor compaction 2023-02-13 16:54:18,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@23debefc; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:18,932 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3f0364c8ae730af4bb790813ec085cf7/family1 in testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:54:18,933 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/8a71400a2d5c462f85e199c28ced9e1f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/bf4f5c8cbadc4656aa45e980d512e1b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/82533a170cf74170942c2961ecdaa7ec, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ebe7be09c8fc4e59a173922c7af213f7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/9da48c045e7b426d87c79342302938f4, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/765535e91b8b45639a2a4e447766cfcd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/98ba8ee31fe04bebadaafaab2ee695fa, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/b7b46058767e47f4927aaec30f2b39a0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ec4f7b2916e049e5808d5853d5549355, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp, totalSize=50.0 K 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 8a71400a2d5c462f85e199c28ced9e1f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting bf4f5c8cbadc4656aa45e980d512e1b0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 82533a170cf74170942c2961ecdaa7ec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting ebe7be09c8fc4e59a173922c7af213f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 9da48c045e7b426d87c79342302938f4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 765535e91b8b45639a2a4e447766cfcd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:54:18,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 98ba8ee31fe04bebadaafaab2ee695fa, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:54:18,934 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting b7b46058767e47f4927aaec30f2b39a0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:54:18,934 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting ec4f7b2916e049e5808d5853d5549355, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:54:18,934 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting d5d8b75cecf94431bab0daf743152330_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:54:18,934 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:18,935 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:18,938 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228394aab5bf817472dbe0331bb4f6cfd91 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228394aab5bf817472dbe0331bb4f6cfd91 2023-02-13 16:54:18,940 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512312814d6ea59ab416e874feafb88e2c110 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512312814d6ea59ab416e874feafb88e2c110 2023-02-13 16:54:18,941 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101eb5a7f72f1f5406c9652fc7086878ae0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101eb5a7f72f1f5406c9652fc7086878ae0 2023-02-13 16:54:18,942 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601034bc069c6d16e45c7aa0ced8a5027b308 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601034bc069c6d16e45c7aa0ced8a5027b308 2023-02-13 16:54:18,943 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-13 16:54:18,943 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 16:54:18,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,944 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,944 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,944 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:18,944 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:19,020 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3f0364c8ae730af4bb790813ec085cf7#family1#compaction#34 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:19,030 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205bd2d0cc1c1c14252805dfe3aa0e6ed54 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205bd2d0cc1c1c14252805dfe3aa0e6ed54 2023-02-13 16:54:19,032 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:19,058 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b48d2b7fe2d4de9b9bf8681085f6fef first=Optional[r1] last=Optional[r2] 2023-02-13 16:54:19,058 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b48d2b7fe2d4de9b9bf8681085f6fef}] 2023-02-13 16:54:19,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b48d2b7fe2d4de9b9bf8681085f6fef for inclusion in 3f0364c8ae730af4bb790813ec085cf7/family1 2023-02-13 16:54:19,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-13 16:54:19,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:19,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:19,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7b48d2b7fe2d4de9b9bf8681085f6fef to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i35gm3eck6v3uko9htjorb898f8gln86114pvdc5keqnuagpd942gjhvevs1n0mi/family1/7b48d2b7fe2d4de9b9bf8681085f6fef 2023-02-13 16:54:19,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i35gm3eck6v3uko9htjorb898f8gln86114pvdc5keqnuagpd942gjhvevs1n0mi/family1/7b48d2b7fe2d4de9b9bf8681085f6fef as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/c10f9c0a53674ac68ee42dc57287051a_SeqId_46_ 2023-02-13 16:54:19,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i35gm3eck6v3uko9htjorb898f8gln86114pvdc5keqnuagpd942gjhvevs1n0mi/family1/7b48d2b7fe2d4de9b9bf8681085f6fef into 3f0364c8ae730af4bb790813ec085cf7/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/c10f9c0a53674ac68ee42dc57287051a_SeqId_46_ - updating store file list. 2023-02-13 16:54:19,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/c10f9c0a53674ac68ee42dc57287051a_SeqId_46_ into 3f0364c8ae730af4bb790813ec085cf7/family1 2023-02-13 16:54:19,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i35gm3eck6v3uko9htjorb898f8gln86114pvdc5keqnuagpd942gjhvevs1n0mi/family1/7b48d2b7fe2d4de9b9bf8681085f6fef into 3f0364c8ae730af4bb790813ec085cf7/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/c10f9c0a53674ac68ee42dc57287051a_SeqId_46_) 2023-02-13 16:54:19,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i35gm3eck6v3uko9htjorb898f8gln86114pvdc5keqnuagpd942gjhvevs1n0mi/family1/7b48d2b7fe2d4de9b9bf8681085f6fef 2023-02-13 16:54:19,078 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:19,079 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:19,080 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120177eec562c9004558ac208ca7cb1dfd8e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120177eec562c9004558ac208ca7cb1dfd8e 2023-02-13 16:54:19,081 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205dc18c1d2487c4aa0b444a0c20f358895 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205dc18c1d2487c4aa0b444a0c20f358895 2023-02-13 16:54:19,081 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-13 16:54:19,081 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 16:54:19,082 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:19,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:19,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:19,084 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:19,085 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 690 msec 2023-02-13 16:54:19,219 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=256 (was 249) - Thread LEAK? -, OpenFileDescriptor=613 (was 599) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=299 (was 336), ProcessCount=168 (was 168), AvailableMemoryMB=2039 (was 1998) - AvailableMemoryMB LEAK? - 2023-02-13 16:54:19,229 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=256, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=299, ProcessCount=168, AvailableMemoryMB=2038 2023-02-13 16:54:19,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 16:54:19,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:19,235 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:19,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-13 16:54:19,235 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:19,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 16:54:19,238 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941 2023-02-13 16:54:19,238 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:19,238 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:19,238 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd empty. 2023-02-13 16:54:19,238 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941 empty. 2023-02-13 16:54:19,239 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b empty. 2023-02-13 16:54:19,239 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:19,239 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:19,239 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941 2023-02-13 16:54:19,239 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-13 16:54:19,274 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:19,279 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => 3c2d6fb96f5457e331a804073866942b, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:19,279 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => e794949a1013ec890417b7ad56401941, NAME => 'testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:19,279 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 59af009f928b299d440fa7c710c3bdbd, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:19,298 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:19,298 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing 3c2d6fb96f5457e331a804073866942b, disabling compactions & flushes 2023-02-13 16:54:19,298 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:19,298 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:19,298 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. after waiting 0 ms 2023-02-13 16:54:19,298 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:19,298 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:19,298 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for 3c2d6fb96f5457e331a804073866942b: Waiting for close lock at 1676307259298Disabling compacts and flushes for region at 1676307259298Disabling writes for close at 1676307259298Writing region close event to WAL at 1676307259298Closed at 1676307259298 2023-02-13 16:54:19,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 16:54:19,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 16:54:19,699 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:19,699 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing e794949a1013ec890417b7ad56401941, disabling compactions & flushes 2023-02-13 16:54:19,699 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:19,699 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:19,699 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. after waiting 0 ms 2023-02-13 16:54:19,699 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:19,699 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:19,699 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for e794949a1013ec890417b7ad56401941: Waiting for close lock at 1676307259699Disabling compacts and flushes for region at 1676307259699Disabling writes for close at 1676307259699Writing region close event to WAL at 1676307259699Closed at 1676307259699 2023-02-13 16:54:19,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:19,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 59af009f928b299d440fa7c710c3bdbd, disabling compactions & flushes 2023-02-13 16:54:19,700 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:19,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:19,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. after waiting 0 ms 2023-02-13 16:54:19,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:19,700 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:19,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 59af009f928b299d440fa7c710c3bdbd: Waiting for close lock at 1676307259700Disabling compacts and flushes for region at 1676307259700Disabling writes for close at 1676307259700Writing region close event to WAL at 1676307259700Closed at 1676307259700 2023-02-13 16:54:19,704 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:19,705 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307259704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307259704"}]},"ts":"1676307259704"} 2023-02-13 16:54:19,705 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307259704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307259704"}]},"ts":"1676307259704"} 2023-02-13 16:54:19,705 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307259704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307259704"}]},"ts":"1676307259704"} 2023-02-13 16:54:19,714 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 16:54:19,715 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:19,715 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307259715"}]},"ts":"1676307259715"} 2023-02-13 16:54:19,717 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-13 16:54:19,721 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e794949a1013ec890417b7ad56401941, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=59af009f928b299d440fa7c710c3bdbd, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3c2d6fb96f5457e331a804073866942b, ASSIGN}] 2023-02-13 16:54:19,723 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=59af009f928b299d440fa7c710c3bdbd, ASSIGN 2023-02-13 16:54:19,723 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3c2d6fb96f5457e331a804073866942b, ASSIGN 2023-02-13 16:54:19,724 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e794949a1013ec890417b7ad56401941, ASSIGN 2023-02-13 16:54:19,724 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=59af009f928b299d440fa7c710c3bdbd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:19,725 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e794949a1013ec890417b7ad56401941, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:19,725 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3c2d6fb96f5457e331a804073866942b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:19,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 16:54:19,849 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/.tmp/family1/a861985e68aa42988ae2a1ce98d41a9a as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/a861985e68aa42988ae2a1ce98d41a9a 2023-02-13 16:54:19,855 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 3f0364c8ae730af4bb790813ec085cf7/family1 of 3f0364c8ae730af4bb790813ec085cf7 into a861985e68aa42988ae2a1ce98d41a9a(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:19,855 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3f0364c8ae730af4bb790813ec085cf7: 2023-02-13 16:54:19,856 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676307258932; duration=0sec 2023-02-13 16:54:19,856 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:19,876 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=e794949a1013ec890417b7ad56401941, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:19,876 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=59af009f928b299d440fa7c710c3bdbd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:19,876 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=3c2d6fb96f5457e331a804073866942b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:19,876 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307259876"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307259876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307259876"}]},"ts":"1676307259876"} 2023-02-13 16:54:19,876 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307259876"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307259876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307259876"}]},"ts":"1676307259876"} 2023-02-13 16:54:19,876 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307259876"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307259876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307259876"}]},"ts":"1676307259876"} 2023-02-13 16:54:19,878 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure e794949a1013ec890417b7ad56401941, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:19,879 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=29, state=RUNNABLE; OpenRegionProcedure 59af009f928b299d440fa7c710c3bdbd, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:19,879 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure 3c2d6fb96f5457e331a804073866942b, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:19,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/8a71400a2d5c462f85e199c28ced9e1f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/bf4f5c8cbadc4656aa45e980d512e1b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/82533a170cf74170942c2961ecdaa7ec, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ebe7be09c8fc4e59a173922c7af213f7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/9da48c045e7b426d87c79342302938f4, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/765535e91b8b45639a2a4e447766cfcd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/98ba8ee31fe04bebadaafaab2ee695fa, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/b7b46058767e47f4927aaec30f2b39a0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ec4f7b2916e049e5808d5853d5549355, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_] to archive 2023-02-13 16:54:19,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:19,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/8a71400a2d5c462f85e199c28ced9e1f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/8a71400a2d5c462f85e199c28ced9e1f 2023-02-13 16:54:19,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/bf4f5c8cbadc4656aa45e980d512e1b0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/bf4f5c8cbadc4656aa45e980d512e1b0 2023-02-13 16:54:19,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/82533a170cf74170942c2961ecdaa7ec to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/82533a170cf74170942c2961ecdaa7ec 2023-02-13 16:54:19,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ebe7be09c8fc4e59a173922c7af213f7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ebe7be09c8fc4e59a173922c7af213f7 2023-02-13 16:54:19,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/9da48c045e7b426d87c79342302938f4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/9da48c045e7b426d87c79342302938f4 2023-02-13 16:54:19,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/765535e91b8b45639a2a4e447766cfcd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/765535e91b8b45639a2a4e447766cfcd 2023-02-13 16:54:19,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/98ba8ee31fe04bebadaafaab2ee695fa to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/98ba8ee31fe04bebadaafaab2ee695fa 2023-02-13 16:54:19,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/b7b46058767e47f4927aaec30f2b39a0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/b7b46058767e47f4927aaec30f2b39a0 2023-02-13 16:54:19,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ec4f7b2916e049e5808d5853d5549355 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/ec4f7b2916e049e5808d5853d5549355 2023-02-13 16:54:19,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/family1/d5d8b75cecf94431bab0daf743152330_SeqId_44_ 2023-02-13 16:54:20,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 59af009f928b299d440fa7c710c3bdbd, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 16:54:20,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:20,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,036 INFO [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,037 DEBUG [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd/family1 2023-02-13 16:54:20,038 DEBUG [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd/family1 2023-02-13 16:54:20,038 INFO [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 59af009f928b299d440fa7c710c3bdbd columnFamilyName family1 2023-02-13 16:54:20,040 INFO [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] regionserver.HStore(310): Store=59af009f928b299d440fa7c710c3bdbd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:20,040 INFO [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,042 DEBUG [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd/family2 2023-02-13 16:54:20,042 DEBUG [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd/family2 2023-02-13 16:54:20,042 INFO [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 59af009f928b299d440fa7c710c3bdbd columnFamilyName family2 2023-02-13 16:54:20,044 INFO [StoreOpener-59af009f928b299d440fa7c710c3bdbd-1] regionserver.HStore(310): Store=59af009f928b299d440fa7c710c3bdbd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:20,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:20,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:54:20,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:20,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 59af009f928b299d440fa7c710c3bdbd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74644474, jitterRate=0.11228933930397034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:20,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 59af009f928b299d440fa7c710c3bdbd: Running coprocessor pre-open hook at 1676307260034Writing region info on filesystem at 1676307260034Initializing all the Stores at 1676307260035 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307260035Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307260035Cleaning up temporary data from old regions at 1676307260053 (+18 ms)Cleaning up detritus from prior splits at 1676307260054 (+1 ms)Running coprocessor post-open hooks at 1676307260060 (+6 ms)Region opened successfully at 1676307260060 2023-02-13 16:54:20,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd., pid=32, masterSystemTime=1676307260030 2023-02-13 16:54:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,064 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=59af009f928b299d440fa7c710c3bdbd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,064 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676307260064"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307260064"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307260064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307260064"}]},"ts":"1676307260064"} 2023-02-13 16:54:20,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e794949a1013ec890417b7ad56401941, NAME => 'testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 16:54:20,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:20,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,068 INFO [StoreOpener-e794949a1013ec890417b7ad56401941-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,070 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=29 2023-02-13 16:54:20,070 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=29, state=SUCCESS; OpenRegionProcedure 59af009f928b299d440fa7c710c3bdbd, server=jenkins-hbase4.apache.org,33907,1676307219708 in 187 msec 2023-02-13 16:54:20,073 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=59af009f928b299d440fa7c710c3bdbd, ASSIGN in 349 msec 2023-02-13 16:54:20,075 DEBUG [StoreOpener-e794949a1013ec890417b7ad56401941-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941/family1 2023-02-13 16:54:20,075 DEBUG [StoreOpener-e794949a1013ec890417b7ad56401941-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941/family1 2023-02-13 16:54:20,075 INFO [StoreOpener-e794949a1013ec890417b7ad56401941-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e794949a1013ec890417b7ad56401941 columnFamilyName family1 2023-02-13 16:54:20,076 INFO [StoreOpener-e794949a1013ec890417b7ad56401941-1] regionserver.HStore(310): Store=e794949a1013ec890417b7ad56401941/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:20,076 INFO [StoreOpener-e794949a1013ec890417b7ad56401941-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,078 DEBUG [StoreOpener-e794949a1013ec890417b7ad56401941-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941/family2 2023-02-13 16:54:20,078 DEBUG [StoreOpener-e794949a1013ec890417b7ad56401941-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941/family2 2023-02-13 16:54:20,079 INFO [StoreOpener-e794949a1013ec890417b7ad56401941-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e794949a1013ec890417b7ad56401941 columnFamilyName family2 2023-02-13 16:54:20,080 INFO [StoreOpener-e794949a1013ec890417b7ad56401941-1] regionserver.HStore(310): Store=e794949a1013ec890417b7ad56401941/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:20,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:20,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e794949a1013ec890417b7ad56401941 2023-02-13 16:54:20,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:20,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e794949a1013ec890417b7ad56401941; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70129218, jitterRate=0.045006781816482544}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:20,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e794949a1013ec890417b7ad56401941: Running coprocessor pre-open hook at 1676307260066Writing region info on filesystem at 1676307260066Initializing all the Stores at 1676307260067 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307260067Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307260068 (+1 ms)Cleaning up temporary data from old regions at 1676307260082 (+14 ms)Cleaning up detritus from prior splits at 1676307260083 (+1 ms)Running coprocessor post-open hooks at 1676307260087 (+4 ms)Region opened successfully at 1676307260087 2023-02-13 16:54:20,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941., pid=31, masterSystemTime=1676307260030 2023-02-13 16:54:20,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:20,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3c2d6fb96f5457e331a804073866942b, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 16:54:20,091 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=e794949a1013ec890417b7ad56401941, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,091 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307260090"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307260090"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307260090"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307260090"}]},"ts":"1676307260090"} 2023-02-13 16:54:20,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:20,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 16:54:20,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure e794949a1013ec890417b7ad56401941, server=jenkins-hbase4.apache.org,33907,1676307219708 in 217 msec 2023-02-13 16:54:20,098 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e794949a1013ec890417b7ad56401941, ASSIGN in 376 msec 2023-02-13 16:54:20,104 INFO [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,106 DEBUG [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1 2023-02-13 16:54:20,106 DEBUG [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1 2023-02-13 16:54:20,107 INFO [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c2d6fb96f5457e331a804073866942b columnFamilyName family1 2023-02-13 16:54:20,109 INFO [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] regionserver.HStore(310): Store=3c2d6fb96f5457e331a804073866942b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:20,109 INFO [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,113 DEBUG [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family2 2023-02-13 16:54:20,113 DEBUG [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family2 2023-02-13 16:54:20,113 INFO [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c2d6fb96f5457e331a804073866942b columnFamilyName family2 2023-02-13 16:54:20,115 INFO [StoreOpener-3c2d6fb96f5457e331a804073866942b-1] regionserver.HStore(310): Store=3c2d6fb96f5457e331a804073866942b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:20,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:54:20,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:54:20,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:20,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3c2d6fb96f5457e331a804073866942b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68088945, jitterRate=0.014604344964027405}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:54:20,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3c2d6fb96f5457e331a804073866942b: Running coprocessor pre-open hook at 1676307260091Writing region info on filesystem at 1676307260091Initializing all the Stores at 1676307260094 (+3 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307260094Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307260098 (+4 ms)Cleaning up temporary data from old regions at 1676307260117 (+19 ms)Cleaning up detritus from prior splits at 1676307260118 (+1 ms)Running coprocessor post-open hooks at 1676307260122 (+4 ms)Region opened successfully at 1676307260122 2023-02-13 16:54:20,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b., pid=33, masterSystemTime=1676307260030 2023-02-13 16:54:20,125 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=3c2d6fb96f5457e331a804073866942b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,126 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676307260125"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307260125"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307260125"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307260125"}]},"ts":"1676307260125"} 2023-02-13 16:54:20,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:20,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:20,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-02-13 16:54:20,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure 3c2d6fb96f5457e331a804073866942b, server=jenkins-hbase4.apache.org,33907,1676307219708 in 250 msec 2023-02-13 16:54:20,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-02-13 16:54:20,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3c2d6fb96f5457e331a804073866942b, ASSIGN in 411 msec 2023-02-13 16:54:20,135 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:20,135 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307260135"}]},"ts":"1676307260135"} 2023-02-13 16:54:20,136 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-13 16:54:20,138 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:20,140 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 906 msec 2023-02-13 16:54:20,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 16:54:20,341 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-13 16:54:20,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:20,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,354 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-13 16:54:20,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:20,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-13 16:54:20,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-13 16:54:20,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,355 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,356 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:20,356 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:20,357 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,357 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:20,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:20,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,358 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,362 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-13 16:54:20,362 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:20,362 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-13 16:54:20,362 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,362 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. started... 2023-02-13 16:54:20,368 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 59af009f928b299d440fa7c710c3bdbd: 2023-02-13 16:54:20,368 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,369 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,369 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. started... 2023-02-13 16:54:20,369 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:54:20,369 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:20,369 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:20,369 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:54:20,370 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e794949a1013ec890417b7ad56401941: 2023-02-13 16:54:20,370 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. started... 2023-02-13 16:54:20,371 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,371 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3c2d6fb96f5457e331a804073866942b 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-13 16:54:20,371 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:54:20,394 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021335821d6e1baa4a21b1657b7055af0c5b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021335821d6e1baa4a21b1657b7055af0c5b 2023-02-13 16:54:20,802 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/.tmp/family1/fc181b284e684693a140c042621e0da2 2023-02-13 16:54:20,807 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/.tmp/family1/fc181b284e684693a140c042621e0da2 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/fc181b284e684693a140c042621e0da2 2023-02-13 16:54:20,812 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/fc181b284e684693a140c042621e0da2, entries=1, sequenceid=5, filesize=4.9 K 2023-02-13 16:54:20,814 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 3c2d6fb96f5457e331a804073866942b in 443ms, sequenceid=5, compaction requested=false 2023-02-13 16:54:20,814 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-13 16:54:20,814 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3c2d6fb96f5457e331a804073866942b: 2023-02-13 16:54:20,814 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:20,814 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:54:20,814 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:54:20,814 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,815 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-13 16:54:20,815 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-13 16:54:20,816 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:20,817 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,817 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:20,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:20,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:20,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:20,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-13 16:54:20,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5333cec3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:20,818 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-13 16:54:20,818 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,820 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-13 16:54:20,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-13 16:54:20,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:20,820 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,820 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:20,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-13 16:54:20,823 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:20,823 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-13 16:54:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:20,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:20,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:20,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-13 16:54:20,825 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-13 16:54:20,825 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:20,825 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:20,825 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,825 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:20,826 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,826 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:20,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,827 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:20,827 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:20,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:20,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,828 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:20,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-13 16:54:20,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:20,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:20,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:20,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,834 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:20,834 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:20,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:20,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:20,935 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:20,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:20,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:20,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:20,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,942 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-13 16:54:20,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:20,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-13 16:54:20,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-13 16:54:20,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,942 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,944 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:20,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:20,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:20,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:20,944 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,946 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-13 16:54:20,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:20,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-13 16:54:20,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:20,946 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. started... 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. started... 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 59af009f928b299d440fa7c710c3bdbd: 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e794949a1013ec890417b7ad56401941: 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:54:20,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:54:20,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:54:20,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:20,953 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:20,953 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. started... 2023-02-13 16:54:20,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:54:20,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:54:20,953 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3c2d6fb96f5457e331a804073866942b 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-13 16:54:21,371 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021310e7c7d512e340f1b9c1902fd767ff0c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021310e7c7d512e340f1b9c1902fd767ff0c 2023-02-13 16:54:21,777 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/.tmp/family1/39d1940fb1284e179ad6f9b3e8297853 2023-02-13 16:54:21,782 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/.tmp/family1/39d1940fb1284e179ad6f9b3e8297853 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/39d1940fb1284e179ad6f9b3e8297853 2023-02-13 16:54:21,787 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/39d1940fb1284e179ad6f9b3e8297853, entries=2, sequenceid=9, filesize=5.0 K 2023-02-13 16:54:21,788 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 3c2d6fb96f5457e331a804073866942b in 835ms, sequenceid=9, compaction requested=false 2023-02-13 16:54:21,788 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3c2d6fb96f5457e331a804073866942b: 2023-02-13 16:54:21,788 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:21,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:54:21,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:54:21,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-13 16:54:21,789 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-13 16:54:21,791 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,791 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:21,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:21,791 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,791 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:21,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:21,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:21,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:21,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-13 16:54:21,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5697eeea[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:21,793 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-13 16:54:21,793 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,796 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,796 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-13 16:54:21,796 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-13 16:54:21,796 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:21,796 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,796 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:21,797 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,797 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-13 16:54:21,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:21,797 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:21,798 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-13 16:54:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:21,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-13 16:54:21,800 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-13 16:54:21,800 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:21,800 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:21,800 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,800 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:21,802 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,802 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:21,802 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:21,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:21,802 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,803 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:21,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,805 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:21,806 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:21,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:21,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-13 16:54:21,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:21,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:21,806 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:21,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:21,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:21,906 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:21,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:21,915 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-13 16:54:21,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 16:54:21,917 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 16:54:21,918 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:21,918 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307861918 2023-02-13 16:54:21,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:54:21,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-13 16:54:21,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 16:54:21,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x5733651b to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:21,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@700f30a9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:21,925 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-13 16:54:22,366 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302135fbf774c101242d594ed52307d7cd6f6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302135fbf774c101242d594ed52307d7cd6f6 2023-02-13 16:54:22,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 16:54:22,382 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:22,383 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55808, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:22,398 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:22,400 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49400, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:22,418 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/b46d2d364f114bc6b869003ffedc80c9 first=Optional[k0] last=Optional[k1] 2023-02-13 16:54:22,419 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/b46d2d364f114bc6b869003ffedc80c9}] 2023-02-13 16:54:22,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/b46d2d364f114bc6b869003ffedc80c9 for inclusion in 3c2d6fb96f5457e331a804073866942b/family1 2023-02-13 16:54:22,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-13 16:54:22,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 16:54:22,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 3c2d6fb96f5457e331a804073866942b: 2023-02-13 16:54:22,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/b46d2d364f114bc6b869003ffedc80c9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testScannerOnBulkLoadRefHFiles__pbsiop33s3090vm6a5tnbqvool57k214svllfbt9r84alkoct8fm8idm0repqpjn/family1/b46d2d364f114bc6b869003ffedc80c9 2023-02-13 16:54:22,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testScannerOnBulkLoadRefHFiles__pbsiop33s3090vm6a5tnbqvool57k214svllfbt9r84alkoct8fm8idm0repqpjn/family1/b46d2d364f114bc6b869003ffedc80c9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_ 2023-02-13 16:54:22,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testScannerOnBulkLoadRefHFiles__pbsiop33s3090vm6a5tnbqvool57k214svllfbt9r84alkoct8fm8idm0repqpjn/family1/b46d2d364f114bc6b869003ffedc80c9 into 3c2d6fb96f5457e331a804073866942b/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_ - updating store file list. 2023-02-13 16:54:22,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_ into 3c2d6fb96f5457e331a804073866942b/family1 2023-02-13 16:54:22,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testScannerOnBulkLoadRefHFiles__pbsiop33s3090vm6a5tnbqvool57k214svllfbt9r84alkoct8fm8idm0repqpjn/family1/b46d2d364f114bc6b869003ffedc80c9 into 3c2d6fb96f5457e331a804073866942b/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_) 2023-02-13 16:54:22,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testScannerOnBulkLoadRefHFiles__pbsiop33s3090vm6a5tnbqvool57k214svllfbt9r84alkoct8fm8idm0repqpjn/family1/b46d2d364f114bc6b869003ffedc80c9 2023-02-13 16:54:22,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:54:22,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:54:22,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): 3c2d6fb96f5457e331a804073866942b/family1 is initiating minor compaction (all files) 2023-02-13 16:54:22,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6ce5d05a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:22,436 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3c2d6fb96f5457e331a804073866942b/family1 in testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:54:22,436 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/fc181b284e684693a140c042621e0da2, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/39d1940fb1284e179ad6f9b3e8297853, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/.tmp, totalSize=15.1 K 2023-02-13 16:54:22,437 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting fc181b284e684693a140c042621e0da2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307260342 2023-02-13 16:54:22,438 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 39d1940fb1284e179ad6f9b3e8297853, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307260342 2023-02-13 16:54:22,438 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 9690dfd1c3884e74b3684f18464a58d7_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307260342 2023-02-13 16:54:22,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 16:54:22,441 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:22,443 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021310e7c7d512e340f1b9c1902fd767ff0c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021310e7c7d512e340f1b9c1902fd767ff0c 2023-02-13 16:54:22,444 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021335821d6e1baa4a21b1657b7055af0c5b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021335821d6e1baa4a21b1657b7055af0c5b 2023-02-13 16:54:22,444 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-13 16:54:22,444 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 16:54:22,445 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-13 16:54:22,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 16:54:22,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:22,446 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 16:54:22,448 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 531 msec 2023-02-13 16:54:22,460 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3c2d6fb96f5457e331a804073866942b#family1#compaction#37 average throughput is 0.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:22,479 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=259 (was 256) - Thread LEAK? -, OpenFileDescriptor=620 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=283 (was 299), ProcessCount=168 (was 168), AvailableMemoryMB=1760 (was 2038) 2023-02-13 16:54:22,490 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=258, OpenFileDescriptor=618, MaxFileDescriptor=60000, SystemLoadAverage=283, ProcessCount=168, AvailableMemoryMB=1759 2023-02-13 16:54:22,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 16:54:22,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:22,494 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:22,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-13 16:54:22,495 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:22,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:22,496 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:22,496 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872 empty. 2023-02-13 16:54:22,497 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:22,497 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-13 16:54:22,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:22,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:22,904 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:22,907 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => f4c3c0edca48a44c27c6f10e06a93872, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:22,914 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:22,915 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing f4c3c0edca48a44c27c6f10e06a93872, disabling compactions & flushes 2023-02-13 16:54:22,915 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:22,915 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:22,915 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. after waiting 0 ms 2023-02-13 16:54:22,915 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:22,915 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:22,915 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for f4c3c0edca48a44c27c6f10e06a93872: Waiting for close lock at 1676307262915Disabling compacts and flushes for region at 1676307262915Disabling writes for close at 1676307262915Writing region close event to WAL at 1676307262915Closed at 1676307262915 2023-02-13 16:54:22,917 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:22,918 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307262918"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307262918"}]},"ts":"1676307262918"} 2023-02-13 16:54:22,919 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:22,920 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:22,920 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307262920"}]},"ts":"1676307262920"} 2023-02-13 16:54:22,921 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-13 16:54:22,925 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f4c3c0edca48a44c27c6f10e06a93872, ASSIGN}] 2023-02-13 16:54:22,927 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f4c3c0edca48a44c27c6f10e06a93872, ASSIGN 2023-02-13 16:54:22,927 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f4c3c0edca48a44c27c6f10e06a93872, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:23,078 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f4c3c0edca48a44c27c6f10e06a93872, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:23,079 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307263078"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307263078"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307263078"}]},"ts":"1676307263078"} 2023-02-13 16:54:23,080 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure f4c3c0edca48a44c27c6f10e06a93872, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:23,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:23,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:23,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f4c3c0edca48a44c27c6f10e06a93872, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:23,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:23,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,237 INFO [StoreOpener-f4c3c0edca48a44c27c6f10e06a93872-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,239 DEBUG [StoreOpener-f4c3c0edca48a44c27c6f10e06a93872-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1 2023-02-13 16:54:23,239 DEBUG [StoreOpener-f4c3c0edca48a44c27c6f10e06a93872-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1 2023-02-13 16:54:23,239 INFO [StoreOpener-f4c3c0edca48a44c27c6f10e06a93872-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f4c3c0edca48a44c27c6f10e06a93872 columnFamilyName family1 2023-02-13 16:54:23,241 INFO [StoreOpener-f4c3c0edca48a44c27c6f10e06a93872-1] regionserver.HStore(310): Store=f4c3c0edca48a44c27c6f10e06a93872/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:23,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:54:23,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:23,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f4c3c0edca48a44c27c6f10e06a93872; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68054453, jitterRate=0.014090374112129211}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:23,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f4c3c0edca48a44c27c6f10e06a93872: Running coprocessor pre-open hook at 1676307263236Writing region info on filesystem at 1676307263236Initializing all the Stores at 1676307263237 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676307263237Cleaning up temporary data from old regions at 1676307263243 (+6 ms)Cleaning up detritus from prior splits at 1676307263243Running coprocessor post-open hooks at 1676307263250 (+7 ms)Region opened successfully at 1676307263250 2023-02-13 16:54:23,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872., pid=37, masterSystemTime=1676307263232 2023-02-13 16:54:23,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:23,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:23,253 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f4c3c0edca48a44c27c6f10e06a93872, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:23,253 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676307263253"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307263253"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307263253"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307263253"}]},"ts":"1676307263253"} 2023-02-13 16:54:23,257 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-13 16:54:23,257 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure f4c3c0edca48a44c27c6f10e06a93872, server=jenkins-hbase4.apache.org,33907,1676307219708 in 175 msec 2023-02-13 16:54:23,259 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 16:54:23,259 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f4c3c0edca48a44c27c6f10e06a93872, ASSIGN in 332 msec 2023-02-13 16:54:23,260 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:23,260 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307263260"}]},"ts":"1676307263260"} 2023-02-13 16:54:23,261 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-13 16:54:23,266 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:23,267 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 774 msec 2023-02-13 16:54:23,291 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/.tmp/family1/d6514ae58c01453085a1154b966a46a0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/d6514ae58c01453085a1154b966a46a0 2023-02-13 16:54:23,296 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3c2d6fb96f5457e331a804073866942b/family1 of 3c2d6fb96f5457e331a804073866942b into d6514ae58c01453085a1154b966a46a0(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:23,296 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3c2d6fb96f5457e331a804073866942b: 2023-02-13 16:54:23,296 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1676307262436; duration=0sec 2023-02-13 16:54:23,297 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:23,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/fc181b284e684693a140c042621e0da2, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/39d1940fb1284e179ad6f9b3e8297853, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_] to archive 2023-02-13 16:54:23,323 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:23,325 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/fc181b284e684693a140c042621e0da2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/fc181b284e684693a140c042621e0da2 2023-02-13 16:54:23,326 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/39d1940fb1284e179ad6f9b3e8297853 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/39d1940fb1284e179ad6f9b3e8297853 2023-02-13 16:54:23,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/family1/9690dfd1c3884e74b3684f18464a58d7_SeqId_12_ 2023-02-13 16:54:23,379 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:23,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 16:54:23,600 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-13 16:54:23,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:23,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,635 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:23,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:23,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:23,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:23,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,637 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:23,637 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:23,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:23,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:23,637 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:23,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,637 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:23,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:23,638 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:23,638 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:23,638 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:23,639 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:23,639 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:23,639 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:23,639 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:23,639 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:54:24,075 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302678895b179c44ebbe95d0502b0ae849 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511302678895b179c44ebbe95d0502b0ae849 2023-02-13 16:54:24,079 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/865b22db31ed4cf5bc7b9494cf7819bc 2023-02-13 16:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/865b22db31ed4cf5bc7b9494cf7819bc as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/865b22db31ed4cf5bc7b9494cf7819bc 2023-02-13 16:54:24,088 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/865b22db31ed4cf5bc7b9494cf7819bc, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-13 16:54:24,089 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 450ms, sequenceid=5, compaction requested=false 2023-02-13 16:54:24,089 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:24,089 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:24,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:24,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:24,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,090 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:24,090 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:24,091 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,091 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:24,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:24,091 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,091 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:24,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:24,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:24,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:24,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:24,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b6888b1[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:24,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:24,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,094 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:24,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:24,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:24,095 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,095 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:24,098 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,098 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:24,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:24,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:24,098 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:24,098 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:24,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:24,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:24,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:24,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:24,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:24,101 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:24,101 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:24,101 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:24,101 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,101 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:24,103 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,103 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:24,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:24,103 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:24,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:24,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:24,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:24,104 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:24,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,105 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:24,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:24,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:24,108 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:24,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:24,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:24,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:24,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:24,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:24,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:24,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:24,108 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,109 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,109 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,109 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,109 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:24,109 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:24,209 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:24,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:24,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:24,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:24,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,221 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:24,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:24,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:24,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:24,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,224 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:24,224 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:24,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:24,224 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:24,224 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:24,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:24,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,227 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:24,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:24,227 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:24,227 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:24,227 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:24,227 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:24,227 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:24,227 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:24,227 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:24,227 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:54:24,648 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ec3f90c7991840248ab445f9c7f92b2d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128ec3f90c7991840248ab445f9c7f92b2d 2023-02-13 16:54:25,055 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/a06f8b1564034228959d03e7d51c31b7 2023-02-13 16:54:25,060 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/a06f8b1564034228959d03e7d51c31b7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/a06f8b1564034228959d03e7d51c31b7 2023-02-13 16:54:25,065 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/a06f8b1564034228959d03e7d51c31b7, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:54:25,066 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 839ms, sequenceid=9, compaction requested=false 2023-02-13 16:54:25,066 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:25,066 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:25,066 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:25,066 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:25,066 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,067 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:25,067 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:25,068 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,068 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:25,068 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,068 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:25,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:25,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:25,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f4eecae[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:25,070 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:25,070 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,073 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,073 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:25,074 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,074 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:25,076 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:25,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:25,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:25,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:25,076 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:25,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:25,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:25,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:25,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:25,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:25,078 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:25,078 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:25,078 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:25,078 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,078 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:25,080 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,080 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:25,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,080 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:25,081 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,082 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:25,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:25,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:25,084 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,084 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:25,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:25,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:25,184 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:25,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:25,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:25,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:25,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,191 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:25,191 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:25,191 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:25,191 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:25,191 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,191 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,193 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:25,193 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:25,193 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:25,193 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,194 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:25,194 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:25,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:25,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:25,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:25,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:25,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:25,194 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:25,194 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:25,195 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:25,567 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:25,608 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512017437c8d9e0514a1eb5e9df41f85cb285 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512017437c8d9e0514a1eb5e9df41f85cb285 2023-02-13 16:54:26,015 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5e7550d1e39043f7a033fef43e953dc3 2023-02-13 16:54:26,020 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5e7550d1e39043f7a033fef43e953dc3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5e7550d1e39043f7a033fef43e953dc3 2023-02-13 16:54:26,025 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5e7550d1e39043f7a033fef43e953dc3, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 16:54:26,025 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 830ms, sequenceid=13, compaction requested=true 2023-02-13 16:54:26,025 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:26,025 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:26,028 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,028 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,029 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,029 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:26,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:26,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42c67e52[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:26,030 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:26,030 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,031 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,031 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:26,031 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:26,031 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,031 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:26,031 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:26,033 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,033 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,033 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,033 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:26,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:26,035 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:26,035 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:26,035 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:26,035 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,035 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:26,036 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,036 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,036 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,037 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,038 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:26,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:26,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:26,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:26,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:26,141 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:26,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:26,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:26,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:26,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,148 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:26,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:26,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:26,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:26,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,148 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,150 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:26,150 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:26,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:26,150 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,150 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:26,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:26,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:26,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:26,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:26,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:26,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:26,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:26,151 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:26,151 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:26,151 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:26,565 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205b06eb75fb4184412a1a31c478b5371ce to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b06eb75fb4184412a1a31c478b5371ce 2023-02-13 16:54:26,971 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/e0d44a61abfc4b42a3d16c94fafee3a5 2023-02-13 16:54:26,983 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/e0d44a61abfc4b42a3d16c94fafee3a5 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/e0d44a61abfc4b42a3d16c94fafee3a5 2023-02-13 16:54:26,990 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/e0d44a61abfc4b42a3d16c94fafee3a5, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 16:54:26,991 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 840ms, sequenceid=17, compaction requested=true 2023-02-13 16:54:26,991 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:26,991 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:26,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:26,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:26,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:26,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:26,993 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,993 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:26,994 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,994 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:26,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:26,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bcf385e[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:26,996 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:26,996 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,997 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:26,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:26,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:26,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:26,997 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,997 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:26,999 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:26,999 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:26,999 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:26,999 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:27,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:27,002 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:27,002 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:27,002 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:27,002 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,002 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:27,003 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,003 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,003 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,008 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,009 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,012 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:27,012 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,112 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:27,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:27,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:27,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:27,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,120 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:27,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:27,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:27,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,122 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,122 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,122 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:27,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:27,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:27,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:27,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:27,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:27,127 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:27,127 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:27,127 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:27,547 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512285a34ee857d414d5aacdfae7273f51bfe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512285a34ee857d414d5aacdfae7273f51bfe 2023-02-13 16:54:27,952 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/40b87edfc8ba47ae8134cc1dc9ff9f97 2023-02-13 16:54:27,958 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/40b87edfc8ba47ae8134cc1dc9ff9f97 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/40b87edfc8ba47ae8134cc1dc9ff9f97 2023-02-13 16:54:27,962 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/40b87edfc8ba47ae8134cc1dc9ff9f97, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 16:54:27,963 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 836ms, sequenceid=21, compaction requested=true 2023-02-13 16:54:27,964 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:27,964 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:27,968 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,968 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:27,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@541d1096[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:27,970 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:27,970 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,971 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:27,971 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,971 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:27,973 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,973 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,973 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,973 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:27,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:27,975 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:27,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:27,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:27,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,975 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:27,977 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,977 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,977 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:27,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:27,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:27,977 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,978 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:27,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,981 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,981 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,981 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,982 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:27,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:27,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:27,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:27,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:27,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:27,982 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:27,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:27,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,082 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:28,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:28,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:28,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:28,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,089 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:28,089 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,089 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:28,089 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:28,090 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,090 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,092 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:28,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,094 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:28,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:28,094 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:28,094 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:28,094 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:28,508 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317cba2607a90e4313838226757ea3a72c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512317cba2607a90e4313838226757ea3a72c 2023-02-13 16:54:28,515 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5bf9166dc9604e429272cb974348cf5e 2023-02-13 16:54:28,520 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5bf9166dc9604e429272cb974348cf5e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5bf9166dc9604e429272cb974348cf5e 2023-02-13 16:54:28,525 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5bf9166dc9604e429272cb974348cf5e, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 16:54:28,525 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 431ms, sequenceid=25, compaction requested=true 2023-02-13 16:54:28,526 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:28,526 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:28,526 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:28,526 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:28,526 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,526 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:28,526 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:28,528 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,528 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:28,528 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,528 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:28,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:28,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c4ebd27[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:28,530 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:28,530 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,531 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,531 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:28,531 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:28,531 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:28,531 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,531 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:28,533 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:28,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:28,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:28,534 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:28,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:28,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:28,536 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:28,536 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:28,536 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:28,536 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,536 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:28,538 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,538 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:28,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:28,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,538 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:28,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,539 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:28,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,540 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,545 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,545 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,546 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:28,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:28,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:28,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:28,546 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:28,647 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:28,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:28,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:28,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:28,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,655 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:28,655 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,655 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:28,655 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:28,655 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,655 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,656 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:28,656 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:28,657 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:28,657 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,657 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,658 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,658 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:28,658 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:28,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:28,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:28,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:28,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:28,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:28,658 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:28,658 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:28,659 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:29,077 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ed2d463ee9174901aa6c78392d6de4c6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101ed2d463ee9174901aa6c78392d6de4c6 2023-02-13 16:54:29,482 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5ec74b53af054b88b1a2423d10c00285 2023-02-13 16:54:29,487 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5ec74b53af054b88b1a2423d10c00285 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5ec74b53af054b88b1a2423d10c00285 2023-02-13 16:54:29,492 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5ec74b53af054b88b1a2423d10c00285, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 16:54:29,492 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 833ms, sequenceid=29, compaction requested=true 2023-02-13 16:54:29,493 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:29,493 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:29,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:29,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:29,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:29,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:29,495 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,495 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:29,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:29,495 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,495 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:29,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:29,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d8b7212[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:29,497 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:29,497 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,499 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:29,499 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,499 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:29,500 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:29,500 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:29,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:29,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:29,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:29,503 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:29,503 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:29,503 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:29,503 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,503 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:29,505 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,505 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:29,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:29,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,505 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:29,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:29,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:29,505 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:29,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:29,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,507 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:29,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,509 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,509 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:29,509 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,510 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:29,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:29,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:29,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:29,510 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:29,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:29,610 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:29,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:29,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:29,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:29,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,619 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:29,619 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:29,619 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:29,619 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:29,620 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,620 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,628 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:29,628 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:29,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:29,629 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:29,629 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:29,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:29,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:29,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:29,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:29,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:29,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:29,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:29,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:29,636 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:29,636 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:29,636 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:30,051 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ed03820dc4a047d0941436b770f3e3f1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103ed03820dc4a047d0941436b770f3e3f1 2023-02-13 16:54:30,456 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/6bbdcde5e87244908ebd939ad8bcbac8 2023-02-13 16:54:30,461 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/6bbdcde5e87244908ebd939ad8bcbac8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/6bbdcde5e87244908ebd939ad8bcbac8 2023-02-13 16:54:30,465 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/6bbdcde5e87244908ebd939ad8bcbac8, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 16:54:30,466 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 830ms, sequenceid=33, compaction requested=true 2023-02-13 16:54:30,466 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:30,466 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:30,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:30,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:30,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:30,467 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:30,468 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:30,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:30,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:30,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:30,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:30,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:30,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fd4bc35[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:30,470 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:30,470 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,471 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:30,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:30,472 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,472 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:30,472 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:30,473 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:30,473 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,473 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:30,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:30,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:30,473 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:30,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:30,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:30,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:30,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:30,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:30,476 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:30,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:30,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:30,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,476 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:30,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,479 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:30,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:30,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,480 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:30,483 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:30,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:30,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:30,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 16:54:30,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:30,483 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:30,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:30,484 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:30,584 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:30,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:30,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:30,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:30,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,591 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:30,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:30,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:30,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:30,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,593 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:30,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:30,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:30,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:30,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:30,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:30,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,594 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:30,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:30,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:30,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:30,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:30,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:30,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:30,594 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:30,594 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:30,595 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:31,008 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f98bd283978f40569a1dfe3ede6864b1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907f98bd283978f40569a1dfe3ede6864b1 2023-02-13 16:54:31,415 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5568df3610764bdb89f0c10d2c223621 2023-02-13 16:54:31,419 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/5568df3610764bdb89f0c10d2c223621 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5568df3610764bdb89f0c10d2c223621 2023-02-13 16:54:31,423 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5568df3610764bdb89f0c10d2c223621, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 16:54:31,424 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 830ms, sequenceid=37, compaction requested=true 2023-02-13 16:54:31,424 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:31,424 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:31,424 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:31,424 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:31,424 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,424 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:31,424 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:31,426 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,427 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:31,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:31,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@337184f9[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:31,428 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:31,428 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,431 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:31,431 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,431 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:31,431 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:31,432 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:31,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:31,434 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:31,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:31,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:31,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,434 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:31,436 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,436 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:31,436 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,436 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:31,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,437 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:31,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,440 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:31,440 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:31,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:31,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:31,540 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:31,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:31,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:31,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:31,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,548 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:31,548 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,548 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 16:54:31,548 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 16:54:31,548 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,548 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,551 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,551 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:31,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:31,552 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:31,552 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:31,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,553 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 16:54:31,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 16:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 16:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:31,554 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:31,554 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. started... 2023-02-13 16:54:31,554 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4c3c0edca48a44c27c6f10e06a93872 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:31,971 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511208b0711a4862a4ac79950d6df1b3347a8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511208b0711a4862a4ac79950d6df1b3347a8 2023-02-13 16:54:32,376 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/c1c9f17cc0ef436e828a335cd2791a1e 2023-02-13 16:54:32,382 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/c1c9f17cc0ef436e828a335cd2791a1e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/c1c9f17cc0ef436e828a335cd2791a1e 2023-02-13 16:54:32,386 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/c1c9f17cc0ef436e828a335cd2791a1e, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 16:54:32,387 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4c3c0edca48a44c27c6f10e06a93872 in 833ms, sequenceid=41, compaction requested=true 2023-02-13 16:54:32,387 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:32,387 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:32,387 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:32,387 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:32,387 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,387 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 16:54:32,387 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 16:54:32,389 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,389 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,389 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,389 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 16:54:32,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a0fa035[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:32,391 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 16:54:32,391 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,392 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,392 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 16:54:32,392 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 16:54:32,392 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:32,392 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,392 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:32,394 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,394 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 16:54:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 16:54:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:32,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:54:32,397 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 16:54:32,397 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:32,397 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:32,397 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,397 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:32,398 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,399 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,399 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:32,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,399 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,400 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:32,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,405 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 16:54:32,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:32,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:32,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:32,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:32,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:32,405 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:32,506 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:32,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:32,508 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-13 16:54:32,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:32,511 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:32,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:32,512 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307872512 2023-02-13 16:54:32,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:54:32,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-13 16:54:32,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:32,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x5276e02b to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:32,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b7518b4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:32,523 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,367 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010343ee55cf2fd145dc8b70b50c98fd4e0b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010343ee55cf2fd145dc8b70b50c98fd4e0b 2023-02-13 16:54:33,368 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:33,383 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:33,385 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39450, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:33,392 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:33,393 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45462, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:33,401 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a75f843898524f7fa4047a6a13b82c8e first=Optional[r3] last=Optional[r6] 2023-02-13 16:54:33,402 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a75f843898524f7fa4047a6a13b82c8e}] 2023-02-13 16:54:33,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a75f843898524f7fa4047a6a13b82c8e for inclusion in f4c3c0edca48a44c27c6f10e06a93872/family1 2023-02-13 16:54:33,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 16:54:33,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:33,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:33,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a75f843898524f7fa4047a6a13b82c8e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__71h5oaq78kj5c28hvru75la6sa60tlpupkbvbui76ev6qsi4pc34e4bn0o8mlj8f/family1/a75f843898524f7fa4047a6a13b82c8e 2023-02-13 16:54:33,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__71h5oaq78kj5c28hvru75la6sa60tlpupkbvbui76ev6qsi4pc34e4bn0o8mlj8f/family1/a75f843898524f7fa4047a6a13b82c8e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_ 2023-02-13 16:54:33,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__71h5oaq78kj5c28hvru75la6sa60tlpupkbvbui76ev6qsi4pc34e4bn0o8mlj8f/family1/a75f843898524f7fa4047a6a13b82c8e into f4c3c0edca48a44c27c6f10e06a93872/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_ - updating store file list. 2023-02-13 16:54:33,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_ into f4c3c0edca48a44c27c6f10e06a93872/family1 2023-02-13 16:54:33,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__71h5oaq78kj5c28hvru75la6sa60tlpupkbvbui76ev6qsi4pc34e4bn0o8mlj8f/family1/a75f843898524f7fa4047a6a13b82c8e into f4c3c0edca48a44c27c6f10e06a93872/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_) 2023-02-13 16:54:33,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__71h5oaq78kj5c28hvru75la6sa60tlpupkbvbui76ev6qsi4pc34e4bn0o8mlj8f/family1/a75f843898524f7fa4047a6a13b82c8e 2023-02-13 16:54:33,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:54:33,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:54:33,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): f4c3c0edca48a44c27c6f10e06a93872/family1 is initiating minor compaction 2023-02-13 16:54:33,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4f71dc1d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:33,420 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f4c3c0edca48a44c27c6f10e06a93872/family1 in testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:33,420 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/a06f8b1564034228959d03e7d51c31b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5e7550d1e39043f7a033fef43e953dc3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/e0d44a61abfc4b42a3d16c94fafee3a5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/40b87edfc8ba47ae8134cc1dc9ff9f97, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5bf9166dc9604e429272cb974348cf5e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5ec74b53af054b88b1a2423d10c00285, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/6bbdcde5e87244908ebd939ad8bcbac8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5568df3610764bdb89f0c10d2c223621, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/c1c9f17cc0ef436e828a335cd2791a1e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp, totalSize=50.0 K 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting a06f8b1564034228959d03e7d51c31b7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5e7550d1e39043f7a033fef43e953dc3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting e0d44a61abfc4b42a3d16c94fafee3a5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 40b87edfc8ba47ae8134cc1dc9ff9f97, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5bf9166dc9604e429272cb974348cf5e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5ec74b53af054b88b1a2423d10c00285, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 6bbdcde5e87244908ebd939ad8bcbac8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5568df3610764bdb89f0c10d2c223621, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting c1c9f17cc0ef436e828a335cd2791a1e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:54:33,420 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:54:33,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:33,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:33,424 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512285a34ee857d414d5aacdfae7273f51bfe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512285a34ee857d414d5aacdfae7273f51bfe 2023-02-13 16:54:33,425 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512317cba2607a90e4313838226757ea3a72c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512317cba2607a90e4313838226757ea3a72c 2023-02-13 16:54:33,426 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101ed2d463ee9174901aa6c78392d6de4c6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101ed2d463ee9174901aa6c78392d6de4c6 2023-02-13 16:54:33,428 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103ed03820dc4a047d0941436b770f3e3f1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103ed03820dc4a047d0941436b770f3e3f1 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-13 16:54:33,428 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:33,456 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f4c3c0edca48a44c27c6f10e06a93872#family1#compaction#48 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:33,876 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/51b2e647a8d14380b612e9595c8eee98 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/51b2e647a8d14380b612e9595c8eee98 2023-02-13 16:54:33,881 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f4c3c0edca48a44c27c6f10e06a93872/family1 of f4c3c0edca48a44c27c6f10e06a93872 into 51b2e647a8d14380b612e9595c8eee98(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:33,881 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:33,881 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676307273420; duration=0sec 2023-02-13 16:54:33,881 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:33,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/a06f8b1564034228959d03e7d51c31b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5e7550d1e39043f7a033fef43e953dc3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/e0d44a61abfc4b42a3d16c94fafee3a5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/40b87edfc8ba47ae8134cc1dc9ff9f97, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5bf9166dc9604e429272cb974348cf5e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5ec74b53af054b88b1a2423d10c00285, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/6bbdcde5e87244908ebd939ad8bcbac8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5568df3610764bdb89f0c10d2c223621, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/c1c9f17cc0ef436e828a335cd2791a1e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_] to archive 2023-02-13 16:54:33,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:33,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/a06f8b1564034228959d03e7d51c31b7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/a06f8b1564034228959d03e7d51c31b7 2023-02-13 16:54:33,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5e7550d1e39043f7a033fef43e953dc3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5e7550d1e39043f7a033fef43e953dc3 2023-02-13 16:54:33,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/e0d44a61abfc4b42a3d16c94fafee3a5 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/e0d44a61abfc4b42a3d16c94fafee3a5 2023-02-13 16:54:33,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/40b87edfc8ba47ae8134cc1dc9ff9f97 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/40b87edfc8ba47ae8134cc1dc9ff9f97 2023-02-13 16:54:33,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5bf9166dc9604e429272cb974348cf5e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5bf9166dc9604e429272cb974348cf5e 2023-02-13 16:54:33,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5ec74b53af054b88b1a2423d10c00285 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5ec74b53af054b88b1a2423d10c00285 2023-02-13 16:54:33,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/6bbdcde5e87244908ebd939ad8bcbac8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/6bbdcde5e87244908ebd939ad8bcbac8 2023-02-13 16:54:33,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5568df3610764bdb89f0c10d2c223621 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/5568df3610764bdb89f0c10d2c223621 2023-02-13 16:54:33,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/c1c9f17cc0ef436e828a335cd2791a1e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/c1c9f17cc0ef436e828a335cd2791a1e 2023-02-13 16:54:33,949 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/aef6b0c9dbc442bf875dca3c075eba17_SeqId_44_ 2023-02-13 16:54:34,283 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120501f6d217ee184b9aa9bcc457dae1615f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120501f6d217ee184b9aa9bcc457dae1615f 2023-02-13 16:54:34,284 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:34,310 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/09b2c26a67ae41dc82ab9956310d6f5f first=Optional[r00] last=Optional[r2] 2023-02-13 16:54:34,310 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/09b2c26a67ae41dc82ab9956310d6f5f}] 2023-02-13 16:54:34,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/09b2c26a67ae41dc82ab9956310d6f5f for inclusion in f4c3c0edca48a44c27c6f10e06a93872/family1 2023-02-13 16:54:34,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 16:54:34,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:34,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:34,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/09b2c26a67ae41dc82ab9956310d6f5f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tddrau6mt0vta2a6ph0pu2kdmoelv28f03dvkfgsribb99u0onums0pa0n3b1oj/family1/09b2c26a67ae41dc82ab9956310d6f5f 2023-02-13 16:54:34,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tddrau6mt0vta2a6ph0pu2kdmoelv28f03dvkfgsribb99u0onums0pa0n3b1oj/family1/09b2c26a67ae41dc82ab9956310d6f5f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_ 2023-02-13 16:54:34,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tddrau6mt0vta2a6ph0pu2kdmoelv28f03dvkfgsribb99u0onums0pa0n3b1oj/family1/09b2c26a67ae41dc82ab9956310d6f5f into f4c3c0edca48a44c27c6f10e06a93872/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_ - updating store file list. 2023-02-13 16:54:34,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_ into f4c3c0edca48a44c27c6f10e06a93872/family1 2023-02-13 16:54:34,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tddrau6mt0vta2a6ph0pu2kdmoelv28f03dvkfgsribb99u0onums0pa0n3b1oj/family1/09b2c26a67ae41dc82ab9956310d6f5f into f4c3c0edca48a44c27c6f10e06a93872/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_) 2023-02-13 16:54:34,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyPolicy__tddrau6mt0vta2a6ph0pu2kdmoelv28f03dvkfgsribb99u0onums0pa0n3b1oj/family1/09b2c26a67ae41dc82ab9956310d6f5f 2023-02-13 16:54:34,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:54:34,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:54:34,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): f4c3c0edca48a44c27c6f10e06a93872/family1 is initiating minor compaction (all files) 2023-02-13 16:54:34,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4a6f2ab0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:34,327 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f4c3c0edca48a44c27c6f10e06a93872/family1 in testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:54:34,327 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/865b22db31ed4cf5bc7b9494cf7819bc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/51b2e647a8d14380b612e9595c8eee98, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp, totalSize=337.0 K 2023-02-13 16:54:34,328 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 865b22db31ed4cf5bc7b9494cf7819bc, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842816 2023-02-13 16:54:34,328 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 51b2e647a8d14380b612e9595c8eee98, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242816 2023-02-13 16:54:34,329 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 63603133be2841f2b996293298b3df2c_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842816 2023-02-13 16:54:34,329 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:34,330 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:34,331 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511302678895b179c44ebbe95d0502b0ae849 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511302678895b179c44ebbe95d0502b0ae849 2023-02-13 16:54:34,332 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512017437c8d9e0514a1eb5e9df41f85cb285 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512017437c8d9e0514a1eb5e9df41f85cb285 2023-02-13 16:54:34,334 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b06eb75fb4184412a1a31c478b5371ce to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b06eb75fb4184412a1a31c478b5371ce 2023-02-13 16:54:34,334 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-13 16:54:34,334 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 16:54:34,334 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 16:54:34,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 16:54:34,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:34,336 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:54:34,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8270 sec 2023-02-13 16:54:34,344 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f4c3c0edca48a44c27c6f10e06a93872#family1#compaction#49 average throughput is 22.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:54:34,435 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=263 (was 258) - Thread LEAK? -, OpenFileDescriptor=630 (was 618) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=263 (was 283), ProcessCount=168 (was 168), AvailableMemoryMB=1501 (was 1759) 2023-02-13 16:54:34,446 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=263, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=263, ProcessCount=168, AvailableMemoryMB=1501 2023-02-13 16:54:34,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 16:54:34,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:34,451 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:34,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-13 16:54:34,451 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:34,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 16:54:34,453 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:34,453 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 empty. 2023-02-13 16:54:34,453 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:34,453 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-13 16:54:34,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 16:54:34,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 16:54:34,862 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:34,864 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => a3c541a8500ba1157e878a28258f5899, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:35,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 16:54:35,168 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/.tmp/family1/696696a744284396a04deee721435231 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/696696a744284396a04deee721435231 2023-02-13 16:54:35,174 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f4c3c0edca48a44c27c6f10e06a93872/family1 of f4c3c0edca48a44c27c6f10e06a93872 into 696696a744284396a04deee721435231(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:35,174 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f4c3c0edca48a44c27c6f10e06a93872: 2023-02-13 16:54:35,174 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1676307274327; duration=0sec 2023-02-13 16:54:35,174 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:35,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/865b22db31ed4cf5bc7b9494cf7819bc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/51b2e647a8d14380b612e9595c8eee98, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_] to archive 2023-02-13 16:54:35,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:35,227 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/865b22db31ed4cf5bc7b9494cf7819bc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/865b22db31ed4cf5bc7b9494cf7819bc 2023-02-13 16:54:35,229 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/51b2e647a8d14380b612e9595c8eee98 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/51b2e647a8d14380b612e9595c8eee98 2023-02-13 16:54:35,231 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/family1/63603133be2841f2b996293298b3df2c_SeqId_47_ 2023-02-13 16:54:35,279 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:35,279 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing a3c541a8500ba1157e878a28258f5899, disabling compactions & flushes 2023-02-13 16:54:35,279 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,279 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,279 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. after waiting 0 ms 2023-02-13 16:54:35,279 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,279 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,279 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for a3c541a8500ba1157e878a28258f5899: Waiting for close lock at 1676307275279Disabling compacts and flushes for region at 1676307275279Disabling writes for close at 1676307275279Writing region close event to WAL at 1676307275279Closed at 1676307275279 2023-02-13 16:54:35,282 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:35,283 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307275283"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307275283"}]},"ts":"1676307275283"} 2023-02-13 16:54:35,285 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:35,285 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:35,286 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307275285"}]},"ts":"1676307275285"} 2023-02-13 16:54:35,287 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-13 16:54:35,290 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, ASSIGN}] 2023-02-13 16:54:35,292 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, ASSIGN 2023-02-13 16:54:35,293 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:35,444 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:35,444 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307275444"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307275444"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307275444"}]},"ts":"1676307275444"} 2023-02-13 16:54:35,446 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:35,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 16:54:35,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3c541a8500ba1157e878a28258f5899, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:35,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:35,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,602 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,603 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:35,604 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:35,604 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3c541a8500ba1157e878a28258f5899 columnFamilyName family1 2023-02-13 16:54:35,605 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(310): Store=a3c541a8500ba1157e878a28258f5899/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:35,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:35,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:35,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a3c541a8500ba1157e878a28258f5899; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63374880, jitterRate=-0.05564069747924805}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:35,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a3c541a8500ba1157e878a28258f5899: Running coprocessor pre-open hook at 1676307275601Writing region info on filesystem at 1676307275601Initializing all the Stores at 1676307275602 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676307275602Cleaning up temporary data from old regions at 1676307275606 (+4 ms)Cleaning up detritus from prior splits at 1676307275607 (+1 ms)Running coprocessor post-open hooks at 1676307275610 (+3 ms)Region opened successfully at 1676307275610 2023-02-13 16:54:35,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., pid=41, masterSystemTime=1676307275598 2023-02-13 16:54:35,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:35,613 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:35,613 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307275613"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307275613"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307275613"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307275613"}]},"ts":"1676307275613"} 2023-02-13 16:54:35,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-13 16:54:35,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708 in 168 msec 2023-02-13 16:54:35,617 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-13 16:54:35,617 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, ASSIGN in 325 msec 2023-02-13 16:54:35,617 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:35,617 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307275617"}]},"ts":"1676307275617"} 2023-02-13 16:54:35,618 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-13 16:54:35,620 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:35,621 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1710 sec 2023-02-13 16:54:36,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 16:54:36,557 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-13 16:54:36,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:36,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,590 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:36,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:36,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:36,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:36,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,591 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,593 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:36,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:36,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:36,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:36,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:36,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:36,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:36,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:36,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:36,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:36,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:36,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:36,594 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:36,594 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:36,594 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:36,595 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:54:36,619 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:54:36,620 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04df23ee to 127.0.0.1:64619 2023-02-13 16:54:36,620 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:54:36,620 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:54:36,620 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5733651b to 127.0.0.1:64619 2023-02-13 16:54:36,620 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:54:36,620 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:54:36,620 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5276e02b to 127.0.0.1:64619 2023-02-13 16:54:36,621 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:54:37,037 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130fd63ca14baa2409d8e72026dfab3b114 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130fd63ca14baa2409d8e72026dfab3b114 2023-02-13 16:54:37,441 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/08d214e2db2c4aa6829511745b6da4da 2023-02-13 16:54:37,447 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/08d214e2db2c4aa6829511745b6da4da as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da 2023-02-13 16:54:37,451 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-13 16:54:37,453 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 858ms, sequenceid=5, compaction requested=false 2023-02-13 16:54:37,453 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:37,453 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:37,453 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:37,453 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:37,453 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:37,454 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,454 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:37,454 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:37,459 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,459 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:37,460 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,460 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:37,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:37,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:37,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c124484[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:37,462 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:37,462 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,463 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:37,464 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,464 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:37,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:37,466 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,466 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:37,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:37,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:37,469 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:37,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:37,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:37,469 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,469 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:37,471 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,471 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:37,471 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:37,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:37,471 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:37,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,475 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:37,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:37,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:37,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 16:54:37,478 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:37,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:37,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:37,478 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:37,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:37,479 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:37,579 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:37,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:37,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:37,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:37,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,589 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:37,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:37,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:37,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:37,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,592 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:37,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:37,592 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:37,593 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:37,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:37,596 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:37,596 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:37,596 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:37,597 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:54:38,018 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280ff3febbea7f42ae997a3f4f81832be3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280ff3febbea7f42ae997a3f4f81832be3 2023-02-13 16:54:38,424 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/bdef498c83194c7588ecb28a907d1879 2023-02-13 16:54:38,430 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/bdef498c83194c7588ecb28a907d1879 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/bdef498c83194c7588ecb28a907d1879 2023-02-13 16:54:38,434 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/bdef498c83194c7588ecb28a907d1879, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:54:38,435 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 838ms, sequenceid=9, compaction requested=false 2023-02-13 16:54:38,436 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:38,436 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:38,436 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:38,436 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:38,436 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,436 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:38,436 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:38,438 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,438 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:38,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@468d7827[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:38,440 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:38,440 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,441 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:38,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:38,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:38,442 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,442 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:38,443 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,443 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:38,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,443 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,443 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:38,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:38,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:38,446 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:38,446 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:38,446 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:38,446 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,446 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:38,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:38,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,449 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:38,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,450 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:38,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,451 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:38,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:38,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,456 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,456 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,457 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,457 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:38,457 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:38,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:38,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:38,557 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:38,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:38,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:38,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:38,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,564 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:38,564 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,564 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:38,564 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:38,565 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,565 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,566 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:38,566 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:38,566 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:38,566 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:38,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:38,567 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:38,567 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:38,567 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:38,568 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:38,568 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:38,568 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:38,568 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:38,568 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:38,980 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201043bfadf1d1e4074a8db835d950d744f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201043bfadf1d1e4074a8db835d950d744f 2023-02-13 16:54:39,388 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/d45ffa9d6f1246e5b4d65f76d7f712b7 2023-02-13 16:54:39,393 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/d45ffa9d6f1246e5b4d65f76d7f712b7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d45ffa9d6f1246e5b4d65f76d7f712b7 2023-02-13 16:54:39,397 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d45ffa9d6f1246e5b4d65f76d7f712b7, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 16:54:39,398 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 830ms, sequenceid=13, compaction requested=true 2023-02-13 16:54:39,398 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:39,398 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:39,398 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:39,398 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:39,398 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,399 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:39,399 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:39,401 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,401 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,401 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,401 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:39,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:39,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79646a43[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:39,403 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:39,403 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,404 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:39,404 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,405 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:39,406 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,407 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:39,407 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,407 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:39,409 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:39,409 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:39,409 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:39,409 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:39,409 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,409 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:39,411 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,411 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,411 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:39,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,411 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:39,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:39,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,413 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,415 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,415 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,415 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,415 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,415 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:39,415 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:39,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:39,416 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,416 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,416 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,416 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:39,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,416 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:39,416 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:39,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:39,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:39,516 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:39,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:39,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:39,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:39,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,522 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:39,522 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:39,522 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:39,522 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:39,523 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,523 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,526 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:39,526 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:39,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:39,526 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:39,526 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:39,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:39,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,527 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,527 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:39,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:39,527 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:39,527 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:39,527 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:39,527 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:39,527 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:39,528 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:39,528 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:39,528 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:39,940 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205eb26c51a6a244e0b9966b474d2dd5be0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb26c51a6a244e0b9966b474d2dd5be0 2023-02-13 16:54:40,345 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/2ab645381e7e4391afc905ac3af92502 2023-02-13 16:54:40,350 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/2ab645381e7e4391afc905ac3af92502 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2ab645381e7e4391afc905ac3af92502 2023-02-13 16:54:40,353 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2ab645381e7e4391afc905ac3af92502, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 16:54:40,354 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 826ms, sequenceid=17, compaction requested=true 2023-02-13 16:54:40,354 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:40,354 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:40,356 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,356 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:40,356 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,356 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:40,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:40,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6efd7352[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:40,358 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:40,358 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,359 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:40,359 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,359 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:40,360 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:40,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:40,362 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:40,362 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:40,362 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:40,362 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,362 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:40,365 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,365 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:40,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,365 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:40,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:40,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,367 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,369 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:40,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:40,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:40,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,369 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:40,370 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:40,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:40,470 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:40,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:40,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:40,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:40,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,476 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:40,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:40,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:40,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:40,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,476 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,477 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,478 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:40,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:40,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:40,478 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:40,478 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:40,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:40,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:40,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:40,479 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:40,479 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:40,479 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:40,479 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:40,479 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:40,479 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:40,479 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:40,479 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:40,891 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288d9a9e9bf06a402299ec49d517bf67b3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512288d9a9e9bf06a402299ec49d517bf67b3 2023-02-13 16:54:41,131 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:41,297 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/dc2fc473e82e46b48d9963ba1b4dabf2 2023-02-13 16:54:41,302 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/dc2fc473e82e46b48d9963ba1b4dabf2 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/dc2fc473e82e46b48d9963ba1b4dabf2 2023-02-13 16:54:41,307 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/dc2fc473e82e46b48d9963ba1b4dabf2, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 16:54:41,308 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 829ms, sequenceid=21, compaction requested=true 2023-02-13 16:54:41,308 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:41,308 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:41,317 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,317 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:41,318 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,318 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10b52c6d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:41,319 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:41,319 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,320 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:41,320 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,320 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:41,322 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:41,322 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:41,322 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:41,322 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:41,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:41,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:41,324 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:41,324 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:41,324 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:41,324 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,324 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:41,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:41,326 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,326 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:41,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:41,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:41,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,327 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,330 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,330 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,330 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:41,331 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:41,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:41,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:41,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:41,331 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:41,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:41,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:41,431 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:41,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:41,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:41,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:41,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,439 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,440 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:41,440 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:41,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:41,441 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:41,441 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:41,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:41,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:41,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:41,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:41,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:41,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:41,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:41,442 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:41,442 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:41,442 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:41,442 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:41,855 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c1ba0ebfb8b94f6db65bdca13304d87d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231c1ba0ebfb8b94f6db65bdca13304d87d 2023-02-13 16:54:42,260 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/2b4d5f4a220e449cb931e46478f1c0b6 2023-02-13 16:54:42,267 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/2b4d5f4a220e449cb931e46478f1c0b6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2b4d5f4a220e449cb931e46478f1c0b6 2023-02-13 16:54:42,271 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2b4d5f4a220e449cb931e46478f1c0b6, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 16:54:42,272 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 830ms, sequenceid=25, compaction requested=true 2023-02-13 16:54:42,272 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:42,272 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:42,272 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:42,272 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:42,272 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,272 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:42,272 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:42,274 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,274 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:42,274 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,274 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:42,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:42,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dd4221c[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:42,276 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:42,276 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,278 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:42,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:42,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:42,278 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,278 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:42,280 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,280 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:42,280 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:42,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:42,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:42,280 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:42,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:42,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:42,283 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:42,283 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:42,283 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:42,284 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,284 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:42,287 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,287 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:42,287 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:42,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:42,288 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:42,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,290 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:42,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:42,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,298 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:42,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:42,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:42,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:42,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:42,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:42,299 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,299 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,299 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,299 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:42,299 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:42,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:42,399 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:42,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:42,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,406 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,408 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,408 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:42,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:42,409 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:42,409 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,410 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:42,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:42,410 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:42,410 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:42,410 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:42,822 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f19755b9ef784fbca3825aaef4d74364 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101f19755b9ef784fbca3825aaef4d74364 2023-02-13 16:54:43,228 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/1126ab95423949afb56ab634d9c317b7 2023-02-13 16:54:43,233 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/1126ab95423949afb56ab634d9c317b7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/1126ab95423949afb56ab634d9c317b7 2023-02-13 16:54:43,237 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/1126ab95423949afb56ab634d9c317b7, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 16:54:43,238 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 828ms, sequenceid=29, compaction requested=true 2023-02-13 16:54:43,238 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:43,238 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:43,238 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:43,238 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:43,238 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,238 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:43,238 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:43,240 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,240 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:43,240 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,240 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:43,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:43,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67386197[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:43,242 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:43,242 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,243 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,243 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:43,243 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,243 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:43,243 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:43,243 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:43,245 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:43,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:43,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:43,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:43,248 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:43,248 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:43,248 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:43,248 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,248 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:43,253 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,253 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,253 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:43,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:43,254 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,255 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,257 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,257 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,257 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:43,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:43,258 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:43,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:43,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 16:54:43,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:43,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:43,258 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,358 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:43,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:43,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:43,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:43,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,365 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:43,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:43,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:43,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:43,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,366 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,366 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:43,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:43,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:43,367 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:43,367 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:43,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:43,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:43,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:43,368 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:43,368 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:43,368 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:43,368 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:43,368 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:43,368 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:43,368 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:43,368 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:43,780 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036c2bb78deb364abe83527795febbb8df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601036c2bb78deb364abe83527795febbb8df 2023-02-13 16:54:44,185 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/5e542968e21c406da7f0fa8fcbe90db7 2023-02-13 16:54:44,190 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/5e542968e21c406da7f0fa8fcbe90db7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/5e542968e21c406da7f0fa8fcbe90db7 2023-02-13 16:54:44,194 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/5e542968e21c406da7f0fa8fcbe90db7, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 16:54:44,195 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 827ms, sequenceid=33, compaction requested=true 2023-02-13 16:54:44,195 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:44,195 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:44,196 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,196 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:44,197 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,197 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:44,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:44,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62daf9c1[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:44,198 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:44,198 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,200 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,200 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:44,200 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:44,200 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:44,200 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,200 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:44,202 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:44,202 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,202 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:44,202 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:44,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:44,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:44,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:44,204 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:44,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:44,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:44,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,204 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:44,206 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,206 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,206 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:44,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:44,207 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:44,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,208 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:44,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:44,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 16:54:44,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:44,210 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:44,210 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:44,210 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:44,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:44,210 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:44,311 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:44,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:44,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:44,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:44,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,317 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:44,317 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:44,317 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:44,317 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:44,318 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,318 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,320 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,320 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:44,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:44,320 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:44,320 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:44,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:44,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:44,322 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:44,322 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:44,322 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:44,322 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:44,733 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907211e9f4599344358991e68cf54876de4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907211e9f4599344358991e68cf54876de4 2023-02-13 16:54:45,138 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/e8fe3975d1454231bc66b5df1d827409 2023-02-13 16:54:45,143 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/e8fe3975d1454231bc66b5df1d827409 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/e8fe3975d1454231bc66b5df1d827409 2023-02-13 16:54:45,147 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/e8fe3975d1454231bc66b5df1d827409, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 16:54:45,148 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 826ms, sequenceid=37, compaction requested=true 2023-02-13 16:54:45,148 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:45,148 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:45,150 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,150 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:45,150 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,150 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:45,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:45,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bb567b3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:45,152 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:45,152 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,153 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,153 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,153 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:45,153 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:45,153 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:45,153 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:45,156 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,156 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:45,156 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:45,156 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:45,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:45,158 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:45,158 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:45,158 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:45,158 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,158 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:45,160 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,160 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:45,160 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:45,160 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:45,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:45,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,161 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:45,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:45,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:45,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:45,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:45,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:45,167 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:45,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:45,167 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:45,267 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:45,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:45,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:45,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:45,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,274 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:45,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:45,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 16:54:45,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 16:54:45,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,277 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:45,277 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:45,277 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,277 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:45,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:45,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 16:54:45,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:45,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 16:54:45,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:45,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 16:54:45,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:45,278 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:45,279 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:45,279 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. started... 2023-02-13 16:54:45,279 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3c541a8500ba1157e878a28258f5899 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:45,691 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e00a91c6c9a04c7680b1a6a0e2d65181 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120e00a91c6c9a04c7680b1a6a0e2d65181 2023-02-13 16:54:46,096 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/ac96f15796944fb7a2a14c32ca514dce 2023-02-13 16:54:46,100 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/ac96f15796944fb7a2a14c32ca514dce as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/ac96f15796944fb7a2a14c32ca514dce 2023-02-13 16:54:46,104 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/ac96f15796944fb7a2a14c32ca514dce, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 16:54:46,105 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3c541a8500ba1157e878a28258f5899 in 826ms, sequenceid=41, compaction requested=true 2023-02-13 16:54:46,105 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:46,105 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:46,105 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:46,105 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:46,105 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,105 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 16:54:46,105 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 16:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:46,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:46,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 16:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47ed7a1b[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:46,109 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 16:54:46,109 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,110 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,110 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 16:54:46,110 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 16:54:46,111 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:46,111 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,111 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:46,112 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 16:54:46,112 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,112 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:46,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:46,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:46,112 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 16:54:46,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:46,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:46,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:46,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:46,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 16:54:46,114 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 16:54:46,114 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:46,114 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:46,114 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,114 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:46,118 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,118 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,118 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:46,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:46,118 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,122 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:46,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 16:54:46,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:46,122 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:46,123 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:46,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:46,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:46,223 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:46,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:46,225 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:46,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:46,227 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:46,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:46,227 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307886227 2023-02-13 16:54:46,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:54:46,230 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 16:54:46,230 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:46,231 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x0182ce51 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:46,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7241c66d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:46,235 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,074 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010329c0e8f217444007a4eac624e0bba326 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010329c0e8f217444007a4eac624e0bba326 2023-02-13 16:54:47,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:47,088 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:47,089 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38762, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:47,096 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:47,098 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38980, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:47,111 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/20bdc10617354f46ac2c178c1cd0af7b first=Optional[r3] last=Optional[r6] 2023-02-13 16:54:47,112 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/20bdc10617354f46ac2c178c1cd0af7b}] 2023-02-13 16:54:47,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/20bdc10617354f46ac2c178c1cd0af7b for inclusion in a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:47,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 16:54:47,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:47,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:47,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/20bdc10617354f46ac2c178c1cd0af7b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qggavue1t1ktd03s4hcqa6irdrtvbqfbjv2ne6k2vaiifvsmvcibfi63c9hmn8g6/family1/20bdc10617354f46ac2c178c1cd0af7b 2023-02-13 16:54:47,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qggavue1t1ktd03s4hcqa6irdrtvbqfbjv2ne6k2vaiifvsmvcibfi63c9hmn8g6/family1/20bdc10617354f46ac2c178c1cd0af7b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_ 2023-02-13 16:54:47,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qggavue1t1ktd03s4hcqa6irdrtvbqfbjv2ne6k2vaiifvsmvcibfi63c9hmn8g6/family1/20bdc10617354f46ac2c178c1cd0af7b into a3c541a8500ba1157e878a28258f5899/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_ - updating store file list. 2023-02-13 16:54:47,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_ into a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:47,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qggavue1t1ktd03s4hcqa6irdrtvbqfbjv2ne6k2vaiifvsmvcibfi63c9hmn8g6/family1/20bdc10617354f46ac2c178c1cd0af7b into a3c541a8500ba1157e878a28258f5899/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_) 2023-02-13 16:54:47,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__qggavue1t1ktd03s4hcqa6irdrtvbqfbjv2ne6k2vaiifvsmvcibfi63c9hmn8g6/family1/20bdc10617354f46ac2c178c1cd0af7b 2023-02-13 16:54:47,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:54:47,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:54:47,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): a3c541a8500ba1157e878a28258f5899/family1 is initiating minor compaction 2023-02-13 16:54:47,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@509b84e4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:47,144 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a3c541a8500ba1157e878a28258f5899/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:47,144 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/bdef498c83194c7588ecb28a907d1879, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d45ffa9d6f1246e5b4d65f76d7f712b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2ab645381e7e4391afc905ac3af92502, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/dc2fc473e82e46b48d9963ba1b4dabf2, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2b4d5f4a220e449cb931e46478f1c0b6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/1126ab95423949afb56ab634d9c317b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/5e542968e21c406da7f0fa8fcbe90db7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/e8fe3975d1454231bc66b5df1d827409, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/ac96f15796944fb7a2a14c32ca514dce, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp, totalSize=50.4 K 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting bdef498c83194c7588ecb28a907d1879, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting d45ffa9d6f1246e5b4d65f76d7f712b7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 2ab645381e7e4391afc905ac3af92502, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting dc2fc473e82e46b48d9963ba1b4dabf2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 2b4d5f4a220e449cb931e46478f1c0b6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 1126ab95423949afb56ab634d9c317b7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5e542968e21c406da7f0fa8fcbe90db7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting e8fe3975d1454231bc66b5df1d827409, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting ac96f15796944fb7a2a14c32ca514dce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:54:47,144 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:54:47,153 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:47,154 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:47,156 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512288d9a9e9bf06a402299ec49d517bf67b3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512288d9a9e9bf06a402299ec49d517bf67b3 2023-02-13 16:54:47,158 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231c1ba0ebfb8b94f6db65bdca13304d87d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231c1ba0ebfb8b94f6db65bdca13304d87d 2023-02-13 16:54:47,159 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101f19755b9ef784fbca3825aaef4d74364 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101f19755b9ef784fbca3825aaef4d74364 2023-02-13 16:54:47,161 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601036c2bb78deb364abe83527795febbb8df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601036c2bb78deb364abe83527795febbb8df 2023-02-13 16:54:47,161 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-13 16:54:47,161 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 16:54:47,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,172 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): a3c541a8500ba1157e878a28258f5899#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:54:47,229 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120546dee8658e9f4e20b2c4c22aa3739a46 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120546dee8658e9f4e20b2c4c22aa3739a46 2023-02-13 16:54:47,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:47,261 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/17f3ec0028264a458dae634b29f52229 first=Optional[r00] last=Optional[r2] 2023-02-13 16:54:47,262 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/17f3ec0028264a458dae634b29f52229}] 2023-02-13 16:54:47,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/17f3ec0028264a458dae634b29f52229 for inclusion in a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:47,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 16:54:47,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:47,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:47,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/17f3ec0028264a458dae634b29f52229 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j88b4v7irqn0ngvcji9s03up3beq80ckf52qprmfdttdp8uuq87urvge7oeuk1m5/family1/17f3ec0028264a458dae634b29f52229 2023-02-13 16:54:47,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j88b4v7irqn0ngvcji9s03up3beq80ckf52qprmfdttdp8uuq87urvge7oeuk1m5/family1/17f3ec0028264a458dae634b29f52229 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_ 2023-02-13 16:54:47,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j88b4v7irqn0ngvcji9s03up3beq80ckf52qprmfdttdp8uuq87urvge7oeuk1m5/family1/17f3ec0028264a458dae634b29f52229 into a3c541a8500ba1157e878a28258f5899/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_ - updating store file list. 2023-02-13 16:54:47,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_ into a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:47,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j88b4v7irqn0ngvcji9s03up3beq80ckf52qprmfdttdp8uuq87urvge7oeuk1m5/family1/17f3ec0028264a458dae634b29f52229 into a3c541a8500ba1157e878a28258f5899/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_) 2023-02-13 16:54:47,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j88b4v7irqn0ngvcji9s03up3beq80ckf52qprmfdttdp8uuq87urvge7oeuk1m5/family1/17f3ec0028264a458dae634b29f52229 2023-02-13 16:54:47,279 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:47,280 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:47,282 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130fd63ca14baa2409d8e72026dfab3b114 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130fd63ca14baa2409d8e72026dfab3b114 2023-02-13 16:54:47,283 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201043bfadf1d1e4074a8db835d950d744f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201043bfadf1d1e4074a8db835d950d744f 2023-02-13 16:54:47,284 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb26c51a6a244e0b9966b474d2dd5be0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb26c51a6a244e0b9966b474d2dd5be0 2023-02-13 16:54:47,284 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-13 16:54:47,284 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 16:54:47,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:47,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:47,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:47,286 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:47,287 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.0610 sec 2023-02-13 16:54:47,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 16:54:47,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:47,598 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/a5ab67a65d954d78b773883a2d6a0d79 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/a5ab67a65d954d78b773883a2d6a0d79 2023-02-13 16:54:47,602 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a3c541a8500ba1157e878a28258f5899/family1 of a3c541a8500ba1157e878a28258f5899 into a5ab67a65d954d78b773883a2d6a0d79(size=6.8 K), total size for store is 390.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:54:47,602 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:47,602 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676307287143; duration=0sec 2023-02-13 16:54:47,603 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:47,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/bdef498c83194c7588ecb28a907d1879, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d45ffa9d6f1246e5b4d65f76d7f712b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2ab645381e7e4391afc905ac3af92502, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/dc2fc473e82e46b48d9963ba1b4dabf2, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2b4d5f4a220e449cb931e46478f1c0b6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/1126ab95423949afb56ab634d9c317b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/5e542968e21c406da7f0fa8fcbe90db7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/e8fe3975d1454231bc66b5df1d827409, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/ac96f15796944fb7a2a14c32ca514dce, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_] to archive 2023-02-13 16:54:47,637 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:47,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/bdef498c83194c7588ecb28a907d1879 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/bdef498c83194c7588ecb28a907d1879 2023-02-13 16:54:47,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d45ffa9d6f1246e5b4d65f76d7f712b7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d45ffa9d6f1246e5b4d65f76d7f712b7 2023-02-13 16:54:47,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2ab645381e7e4391afc905ac3af92502 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2ab645381e7e4391afc905ac3af92502 2023-02-13 16:54:47,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/dc2fc473e82e46b48d9963ba1b4dabf2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/dc2fc473e82e46b48d9963ba1b4dabf2 2023-02-13 16:54:47,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2b4d5f4a220e449cb931e46478f1c0b6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/2b4d5f4a220e449cb931e46478f1c0b6 2023-02-13 16:54:47,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/1126ab95423949afb56ab634d9c317b7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/1126ab95423949afb56ab634d9c317b7 2023-02-13 16:54:47,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/5e542968e21c406da7f0fa8fcbe90db7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/5e542968e21c406da7f0fa8fcbe90db7 2023-02-13 16:54:47,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/e8fe3975d1454231bc66b5df1d827409 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/e8fe3975d1454231bc66b5df1d827409 2023-02-13 16:54:47,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/ac96f15796944fb7a2a14c32ca514dce to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/ac96f15796944fb7a2a14c32ca514dce 2023-02-13 16:54:47,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/788f8e40a8ab4db4bdc8a770809b3cfc_SeqId_44_ 2023-02-13 16:54:47,782 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 16:54:47,783 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:47,783 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 16:54:47,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-13 16:54:47,787 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE}] 2023-02-13 16:54:47,788 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE 2023-02-13 16:54:47,788 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:47,789 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307287788"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307287788"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307287788"}]},"ts":"1676307287788"} 2023-02-13 16:54:47,790 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:47,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:47,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a3c541a8500ba1157e878a28258f5899, disabling compactions & flushes 2023-02-13 16:54:47,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:47,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:47,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. after waiting 0 ms 2023-02-13 16:54:47,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:47,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-13 16:54:47,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:47,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a3c541a8500ba1157e878a28258f5899: Waiting for close lock at 1676307287942Running coprocessor pre-close hooks at 1676307287942Disabling compacts and flushes for region at 1676307287942Disabling writes for close at 1676307287942Writing region close event to WAL at 1676307287943 (+1 ms)Running coprocessor post-close hooks at 1676307287946 (+3 ms)Closed at 1676307287947 (+1 ms) 2023-02-13 16:54:47,947 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a3c541a8500ba1157e878a28258f5899 to self. 2023-02-13 16:54:47,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:47,949 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=CLOSED 2023-02-13 16:54:47,949 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307287949"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307287949"}]},"ts":"1676307287949"} 2023-02-13 16:54:47,951 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 16:54:47,952 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708 in 160 msec 2023-02-13 16:54:47,952 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=true 2023-02-13 16:54:48,102 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:48,103 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307288102"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307288102"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307288102"}]},"ts":"1676307288102"} 2023-02-13 16:54:48,104 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:48,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:48,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3c541a8500ba1157e878a28258f5899, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:48,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:48,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,260 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,261 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:48,261 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:48,262 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3c541a8500ba1157e878a28258f5899 columnFamilyName family1 2023-02-13 16:54:48,267 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da 2023-02-13 16:54:48,270 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/a5ab67a65d954d78b773883a2d6a0d79 2023-02-13 16:54:48,273 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_ 2023-02-13 16:54:48,273 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(310): Store=a3c541a8500ba1157e878a28258f5899/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:48,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:48,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a3c541a8500ba1157e878a28258f5899; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71672589, jitterRate=0.0680048018693924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:48,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a3c541a8500ba1157e878a28258f5899: Running coprocessor pre-open hook at 1676307288259Writing region info on filesystem at 1676307288259Initializing all the Stores at 1676307288260 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676307288260Cleaning up temporary data from old regions at 1676307288275 (+15 ms)Cleaning up detritus from prior splits at 1676307288276 (+1 ms)Running coprocessor post-open hooks at 1676307288279 (+3 ms)Region opened successfully at 1676307288279 2023-02-13 16:54:48,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., pid=47, masterSystemTime=1676307288256 2023-02-13 16:54:48,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:48,280 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:54:48,282 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:54:48,282 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1912): a3c541a8500ba1157e878a28258f5899/family1 is initiating minor compaction (all files) 2023-02-13 16:54:48,282 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a3c541a8500ba1157e878a28258f5899/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:48,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:48,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:48,282 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/a5ab67a65d954d78b773883a2d6a0d79, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp, totalSize=390.1 K 2023-02-13 16:54:48,282 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:48,282 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 08d214e2db2c4aa6829511745b6da4da, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842816 2023-02-13 16:54:48,283 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307288282"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307288282"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307288282"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307288282"}]},"ts":"1676307288282"} 2023-02-13 16:54:48,283 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting a5ab67a65d954d78b773883a2d6a0d79, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242816 2023-02-13 16:54:48,283 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878842816 2023-02-13 16:54:48,286 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-13 16:54:48,286 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708 in 180 msec 2023-02-13 16:54:48,288 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 16:54:48,288 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE in 499 msec 2023-02-13 16:54:48,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-13 16:54:48,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 503 msec 2023-02-13 16:54:48,292 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 919 msec 2023-02-13 16:54:48,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-13 16:54:48,295 INFO [Listener at localhost/36833] compactions.TestMobCompactor(278): alter status finished 2023-02-13 16:54:48,296 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:48,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:48,299 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): a3c541a8500ba1157e878a28258f5899#family1#compaction#61 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:54:48,302 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:48,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:48,303 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307888303 2023-02-13 16:54:48,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:54:48,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 16:54:48,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:48,305 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x0d971b44 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:48,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7199c635, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:48,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:48,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:48,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:48,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:48,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:49,143 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284742932cb1ad4f6eb752d0d25f089ec1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284742932cb1ad4f6eb752d0d25f089ec1 2023-02-13 16:54:49,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:49,156 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:54:49,157 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38772, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:54:49,164 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:54:49,166 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38986, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:54:49,177 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3425e0ea3587404a96871af21db540da first=Optional[r05] last=Optional[r8] 2023-02-13 16:54:49,177 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=50 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3425e0ea3587404a96871af21db540da}] 2023-02-13 16:54:49,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3425e0ea3587404a96871af21db540da for inclusion in a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:49,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 16:54:49,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:54:49,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:49,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3425e0ea3587404a96871af21db540da to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__u8kia31d2s5d9tr7vpjrc4chh3n1vspofavdo0p55nughcrnare53ac3ku6q14q5/family1/3425e0ea3587404a96871af21db540da 2023-02-13 16:54:49,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__u8kia31d2s5d9tr7vpjrc4chh3n1vspofavdo0p55nughcrnare53ac3ku6q14q5/family1/3425e0ea3587404a96871af21db540da as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/47942bfc5be34a8895aeffb7ecb09ef9_SeqId_52_ 2023-02-13 16:54:49,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__u8kia31d2s5d9tr7vpjrc4chh3n1vspofavdo0p55nughcrnare53ac3ku6q14q5/family1/3425e0ea3587404a96871af21db540da into a3c541a8500ba1157e878a28258f5899/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/47942bfc5be34a8895aeffb7ecb09ef9_SeqId_52_ - updating store file list. 2023-02-13 16:54:49,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/47942bfc5be34a8895aeffb7ecb09ef9_SeqId_52_ into a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:49,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__u8kia31d2s5d9tr7vpjrc4chh3n1vspofavdo0p55nughcrnare53ac3ku6q14q5/family1/3425e0ea3587404a96871af21db540da into a3c541a8500ba1157e878a28258f5899/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/47942bfc5be34a8895aeffb7ecb09ef9_SeqId_52_) 2023-02-13 16:54:49,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__u8kia31d2s5d9tr7vpjrc4chh3n1vspofavdo0p55nughcrnare53ac3ku6q14q5/family1/3425e0ea3587404a96871af21db540da 2023-02-13 16:54:49,200 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:49,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:49,203 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120e00a91c6c9a04c7680b1a6a0e2d65181 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120e00a91c6c9a04c7680b1a6a0e2d65181 2023-02-13 16:54:49,205 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280ff3febbea7f42ae997a3f4f81832be3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280ff3febbea7f42ae997a3f4f81832be3 2023-02-13 16:54:49,205 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-13 16:54:49,205 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 16:54:49,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:49,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:49,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:49,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:49,206 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:49,207 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:49,208 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 910 msec 2023-02-13 16:54:49,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 16:54:49,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:49,521 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/.tmp/family1/952832905cad42e1b3a27c22db96462c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/952832905cad42e1b3a27c22db96462c 2023-02-13 16:54:49,526 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a3c541a8500ba1157e878a28258f5899/family1 of a3c541a8500ba1157e878a28258f5899 into 952832905cad42e1b3a27c22db96462c(size=193.0 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:54:49,526 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a3c541a8500ba1157e878a28258f5899: 2023-02-13 16:54:49,526 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., storeName=a3c541a8500ba1157e878a28258f5899/family1, priority=13, startTime=1676307288280; duration=1sec 2023-02-13 16:54:49,526 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:54:49,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/a5ab67a65d954d78b773883a2d6a0d79, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_] to archive 2023-02-13 16:54:49,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:54:49,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/08d214e2db2c4aa6829511745b6da4da 2023-02-13 16:54:49,627 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/a5ab67a65d954d78b773883a2d6a0d79 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/a5ab67a65d954d78b773883a2d6a0d79 2023-02-13 16:54:49,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/d1641c02a36b4ffeacd6221a876aa1ab_SeqId_46_ 2023-02-13 16:54:49,689 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-13 16:54:49,690 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 16:54:49,690 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-13 16:54:49,692 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-13 16:54:49,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE}] 2023-02-13 16:54:49,695 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE 2023-02-13 16:54:49,695 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:49,696 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307289695"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307289695"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307289695"}]},"ts":"1676307289695"} 2023-02-13 16:54:49,697 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:49,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:49,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a3c541a8500ba1157e878a28258f5899, disabling compactions & flushes 2023-02-13 16:54:49,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:49,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:49,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. after waiting 0 ms 2023-02-13 16:54:49,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:49,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-02-13 16:54:49,855 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:49,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a3c541a8500ba1157e878a28258f5899: Waiting for close lock at 1676307289849Running coprocessor pre-close hooks at 1676307289849Disabling compacts and flushes for region at 1676307289849Disabling writes for close at 1676307289849Writing region close event to WAL at 1676307289851 (+2 ms)Running coprocessor post-close hooks at 1676307289855 (+4 ms)Closed at 1676307289855 2023-02-13 16:54:49,855 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a3c541a8500ba1157e878a28258f5899 to self. 2023-02-13 16:54:49,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:49,857 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=CLOSED 2023-02-13 16:54:49,857 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307289857"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307289857"}]},"ts":"1676307289857"} 2023-02-13 16:54:49,859 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-13 16:54:49,859 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708 in 161 msec 2023-02-13 16:54:49,860 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=true 2023-02-13 16:54:50,010 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:50,010 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307290010"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307290010"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307290010"}]},"ts":"1676307290010"} 2023-02-13 16:54:50,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:50,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:50,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3c541a8500ba1157e878a28258f5899, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:50,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:50,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,168 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,169 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:50,169 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1 2023-02-13 16:54:50,169 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3c541a8500ba1157e878a28258f5899 columnFamilyName family1 2023-02-13 16:54:50,174 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/47942bfc5be34a8895aeffb7ecb09ef9_SeqId_52_ 2023-02-13 16:54:50,177 DEBUG [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/family1/952832905cad42e1b3a27c22db96462c 2023-02-13 16:54:50,177 INFO [StoreOpener-a3c541a8500ba1157e878a28258f5899-1] regionserver.HStore(310): Store=a3c541a8500ba1157e878a28258f5899/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:50,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:54:50,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a3c541a8500ba1157e878a28258f5899; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68536223, jitterRate=0.021269306540489197}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:50,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a3c541a8500ba1157e878a28258f5899: Running coprocessor pre-open hook at 1676307290167Writing region info on filesystem at 1676307290167Initializing all the Stores at 1676307290168 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676307290168Cleaning up temporary data from old regions at 1676307290180 (+12 ms)Cleaning up detritus from prior splits at 1676307290180Running coprocessor post-open hooks at 1676307290183 (+3 ms)Region opened successfully at 1676307290183 2023-02-13 16:54:50,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899., pid=53, masterSystemTime=1676307290164 2023-02-13 16:54:50,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:50,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:54:50,185 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3c541a8500ba1157e878a28258f5899, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:50,185 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676307290185"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307290185"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307290185"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307290185"}]},"ts":"1676307290185"} 2023-02-13 16:54:50,188 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-13 16:54:50,188 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure a3c541a8500ba1157e878a28258f5899, server=jenkins-hbase4.apache.org,33907,1676307219708 in 175 msec 2023-02-13 16:54:50,189 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 16:54:50,189 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3c541a8500ba1157e878a28258f5899, REOPEN/MOVE in 494 msec 2023-02-13 16:54:50,191 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 16:54:50,191 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 498 msec 2023-02-13 16:54:50,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 913 msec 2023-02-13 16:54:50,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 16:54:50,194 INFO [Listener at localhost/36833] compactions.TestMobCompactor(278): alter status finished 2023-02-13 16:54:50,195 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 16:54:50,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:50,197 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:50,197 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:50,197 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307890197 2023-02-13 16:54:50,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:54:50,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 16:54:50,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:50,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x57289fe2 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:54:50,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@683f535c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:54:50,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 16:54:50,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 16:54:50,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:54:50,205 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 16:54:50,206 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-02-13 16:54:50,261 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=242 (was 263), OpenFileDescriptor=606 (was 630), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 263), ProcessCount=168 (was 168), AvailableMemoryMB=1309 (was 1501) 2023-02-13 16:54:50,273 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=1309 2023-02-13 16:54:50,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 16:54:50,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:50,277 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:54:50,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-13 16:54:50,278 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:54:50,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 16:54:50,279 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:50,280 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd empty. 2023-02-13 16:54:50,280 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:50,280 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-13 16:54:50,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 16:54:50,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 16:54:50,688 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-13 16:54:50,690 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => 812c35f3f9014f1f8af17e3d2a3991cd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:54:50,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 16:54:51,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:51,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing 812c35f3f9014f1f8af17e3d2a3991cd, disabling compactions & flushes 2023-02-13 16:54:51,104 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. after waiting 0 ms 2023-02-13 16:54:51,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,104 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,104 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for 812c35f3f9014f1f8af17e3d2a3991cd: Waiting for close lock at 1676307291104Disabling compacts and flushes for region at 1676307291104Disabling writes for close at 1676307291104Writing region close event to WAL at 1676307291104Closed at 1676307291104 2023-02-13 16:54:51,107 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:54:51,108 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307291108"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307291108"}]},"ts":"1676307291108"} 2023-02-13 16:54:51,109 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:54:51,110 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:54:51,110 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307291110"}]},"ts":"1676307291110"} 2023-02-13 16:54:51,111 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-13 16:54:51,116 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, ASSIGN}] 2023-02-13 16:54:51,118 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, ASSIGN 2023-02-13 16:54:51,121 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:54:51,272 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:51,272 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307291272"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307291272"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307291272"}]},"ts":"1676307291272"} 2023-02-13 16:54:51,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:54:51,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 16:54:51,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 812c35f3f9014f1f8af17e3d2a3991cd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:54:51,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:54:51,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,430 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,431 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:54:51,431 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:54:51,431 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 812c35f3f9014f1f8af17e3d2a3991cd columnFamilyName family1 2023-02-13 16:54:51,433 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(310): Store=812c35f3f9014f1f8af17e3d2a3991cd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:54:51,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:54:51,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:54:51,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 812c35f3f9014f1f8af17e3d2a3991cd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62301632, jitterRate=-0.07163333892822266}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:54:51,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 812c35f3f9014f1f8af17e3d2a3991cd: Running coprocessor pre-open hook at 1676307291429Writing region info on filesystem at 1676307291429Initializing all the Stores at 1676307291429Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676307291429Cleaning up temporary data from old regions at 1676307291434 (+5 ms)Cleaning up detritus from prior splits at 1676307291435 (+1 ms)Running coprocessor post-open hooks at 1676307291439 (+4 ms)Region opened successfully at 1676307291439 2023-02-13 16:54:51,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., pid=57, masterSystemTime=1676307291425 2023-02-13 16:54:51,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:51,442 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:51,442 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307291442"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307291442"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307291442"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307291442"}]},"ts":"1676307291442"} 2023-02-13 16:54:51,444 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 16:54:51,444 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708 in 169 msec 2023-02-13 16:54:51,445 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-13 16:54:51,446 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, ASSIGN in 328 msec 2023-02-13 16:54:51,446 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:54:51,446 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307291446"}]},"ts":"1676307291446"} 2023-02-13 16:54:51,447 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-13 16:54:51,449 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:54:51,450 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1740 sec 2023-02-13 16:54:52,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 16:54:52,384 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-13 16:54:52,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:52,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,410 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:52,410 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:52,410 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:52,410 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:52,410 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,410 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,413 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:52,413 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:52,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:52,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:52,413 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:52,413 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:52,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:52,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:52,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:52,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:52,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:52,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:52,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:52,414 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:52,414 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:52,415 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:54:52,834 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130e850559942464aa885ffae688e7a54fc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130e850559942464aa885ffae688e7a54fc 2023-02-13 16:54:53,239 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/7db069ed9bf94e2aaef7799d27bfefa7 2023-02-13 16:54:53,244 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/7db069ed9bf94e2aaef7799d27bfefa7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/7db069ed9bf94e2aaef7799d27bfefa7 2023-02-13 16:54:53,248 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/7db069ed9bf94e2aaef7799d27bfefa7, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-13 16:54:53,249 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 834ms, sequenceid=5, compaction requested=false 2023-02-13 16:54:53,249 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:53,249 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:53,249 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:53,249 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:53,249 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:53,249 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,250 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:53,250 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:53,253 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:53,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:53,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:53,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:53,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:53,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:53,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:53,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63fd9fa8[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:53,254 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:53,254 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,255 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:53,255 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,256 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:53,257 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,257 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:53,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:53,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:53,257 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:53,257 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:53,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:53,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:53,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:53,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:53,259 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:53,259 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:53,259 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:53,259 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,259 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:53,260 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,260 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:53,261 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:53,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:53,261 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:53,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,262 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:53,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:53,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:53,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,268 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,268 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:53,269 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:53,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:53,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:53,369 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:53,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:53,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:53,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:53,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,375 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:53,375 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:53,375 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:53,375 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:53,375 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,375 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,377 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,377 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:53,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:53,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:53,377 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:53,377 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:53,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:53,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,378 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:53,378 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:53,378 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:53,378 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:53,378 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:54:53,476 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:54:53,795 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b2d70a0c1b96422b899495ccb9a01218 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b2d70a0c1b96422b899495ccb9a01218 2023-02-13 16:54:54,200 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/b620f31466e647d48207dfe4e6c58764 2023-02-13 16:54:54,204 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/b620f31466e647d48207dfe4e6c58764 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b620f31466e647d48207dfe4e6c58764 2023-02-13 16:54:54,209 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b620f31466e647d48207dfe4e6c58764, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:54:54,209 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 831ms, sequenceid=9, compaction requested=false 2023-02-13 16:54:54,210 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:54,210 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:54,211 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,211 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:54,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:54,212 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,212 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:54,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:54,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:54,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@565c76e7[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:54,213 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:54,213 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,214 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,214 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:54,215 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,215 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:54,215 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:54,215 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:54,217 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:54,219 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:54,219 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:54,219 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:54,219 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,219 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:54,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:54,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:54,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,222 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:54,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,224 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,224 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:54,224 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,224 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:54,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 16:54:54,224 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:54,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:54,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:54,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:54,224 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:54,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,225 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:54,225 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:54,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:54,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:54,325 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:54,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:54,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:54,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:54,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,332 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:54,332 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:54,332 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:54,332 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:54,332 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,332 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,334 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:54,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:54,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:54,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:54,336 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:54,336 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:54,336 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:54,336 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:54,336 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:54,748 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e7aee5960a5647b999344e25b7396334 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201e7aee5960a5647b999344e25b7396334 2023-02-13 16:54:55,153 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/ebc547263d2c4f3d8e92202b192afe64 2023-02-13 16:54:55,158 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/ebc547263d2c4f3d8e92202b192afe64 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/ebc547263d2c4f3d8e92202b192afe64 2023-02-13 16:54:55,161 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/ebc547263d2c4f3d8e92202b192afe64, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 16:54:55,162 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 826ms, sequenceid=13, compaction requested=true 2023-02-13 16:54:55,162 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:55,162 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:55,162 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:55,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:55,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:55,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:55,165 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:55,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:55,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26a41b60[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:55,167 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:55,167 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,169 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:55,169 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,169 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:55,170 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:55,170 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,170 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:55,170 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:55,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:55,172 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:55,172 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:55,172 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:55,172 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,173 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:55,174 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,174 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,174 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:55,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:55,175 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,176 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,184 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,184 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:55,184 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:55,184 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:55,184 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,184 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 16:54:55,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:55,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:55,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:55,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:55,184 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,184 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,185 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,185 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,185 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:55,185 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:55,185 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:55,285 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:55,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:55,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:55,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:55,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,291 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:55,291 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:55,291 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:55,291 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:55,291 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,292 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,293 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,293 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:55,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:55,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:55,293 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:55,293 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:55,295 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:55,295 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:55,295 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:55,295 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:55,295 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:55,295 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:55,708 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205dea04c392669458f8ad3f23d84ba53c0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205dea04c392669458f8ad3f23d84ba53c0 2023-02-13 16:54:56,114 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/dc72aeab8a344d5a9930e28250477a56 2023-02-13 16:54:56,119 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/dc72aeab8a344d5a9930e28250477a56 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/dc72aeab8a344d5a9930e28250477a56 2023-02-13 16:54:56,123 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/dc72aeab8a344d5a9930e28250477a56, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 16:54:56,124 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 829ms, sequenceid=17, compaction requested=true 2023-02-13 16:54:56,124 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:56,124 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:56,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:56,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:56,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:56,124 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:56,126 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,126 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:56,126 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,126 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:56,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:56,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ab5900d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:56,127 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:56,127 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,132 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,132 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:56,132 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:56,132 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:56,132 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,132 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:56,135 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:56,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:56,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:56,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:56,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:56,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:56,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:56,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:56,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:56,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:56,137 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,137 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:56,139 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,139 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:56,139 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:56,140 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,141 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 16:54:56,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:56,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:56,143 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:56,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:56,143 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:56,244 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:56,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:56,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:56,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:56,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,250 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:56,250 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:56,250 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:56,250 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:56,251 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,251 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,252 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,252 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:56,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:56,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:56,252 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:56,252 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:56,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,252 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:56,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:56,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:56,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:56,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:56,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:56,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:56,253 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:56,253 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:56,253 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:56,254 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:56,665 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122849252fa0c3f04905b1d68c11830cfc6a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122849252fa0c3f04905b1d68c11830cfc6a 2023-02-13 16:54:57,070 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/d1b941766af24addac6349a03148b852 2023-02-13 16:54:57,074 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/d1b941766af24addac6349a03148b852 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/d1b941766af24addac6349a03148b852 2023-02-13 16:54:57,078 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/d1b941766af24addac6349a03148b852, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 16:54:57,078 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 824ms, sequenceid=21, compaction requested=true 2023-02-13 16:54:57,079 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:57,079 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:57,082 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:57,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,082 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:57,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:57,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53a5e2c[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:57,084 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:57,084 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,085 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,085 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:57,085 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,085 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:57,085 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:57,085 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:57,087 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,087 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:57,087 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:57,087 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:57,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:57,089 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:57,089 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:57,089 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:57,089 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,089 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:57,091 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,091 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,091 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:57,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:57,091 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:57,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:57,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:57,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:57,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:57,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:57,095 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:57,095 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:57,096 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,096 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,096 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,096 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:57,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:57,096 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:57,096 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:57,196 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:57,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:57,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:57,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:57,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,202 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:57,202 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:57,202 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:57,202 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:57,203 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,203 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,204 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:57,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:57,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:57,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:57,204 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:57,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:57,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:57,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:57,205 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:57,205 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:57,205 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:57,206 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:57,206 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:57,206 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:57,206 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:57,206 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:57,617 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c1903d5d831e403084fa23190268262f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c1903d5d831e403084fa23190268262f 2023-02-13 16:54:58,023 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/44688fbbcdcb45e7bf96661b1d6e7775 2023-02-13 16:54:58,027 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/44688fbbcdcb45e7bf96661b1d6e7775 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/44688fbbcdcb45e7bf96661b1d6e7775 2023-02-13 16:54:58,031 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/44688fbbcdcb45e7bf96661b1d6e7775, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 16:54:58,032 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 826ms, sequenceid=25, compaction requested=true 2023-02-13 16:54:58,032 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:58,032 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:58,032 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:58,032 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:58,032 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,032 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:58,032 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:58,034 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,034 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:58,034 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,034 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30e4fb22[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:58,036 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:58,036 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,037 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:58,037 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,037 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:58,039 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:58,041 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:58,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:58,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:58,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,041 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:58,043 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,043 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:58,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,043 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:58,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:58,043 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,044 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,049 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,049 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:58,049 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:58,049 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:58,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:58,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:58,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:58,049 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:58,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:58,050 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:58,150 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:58,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:58,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:58,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:58,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,156 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:58,156 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:58,156 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:58,156 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:58,157 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,157 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,161 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,161 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:58,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:58,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:58,161 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,161 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,162 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:58,163 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:58,163 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:58,163 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:58,576 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016a9284431ed441a9bc37be071b8a0d26 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601016a9284431ed441a9bc37be071b8a0d26 2023-02-13 16:54:58,982 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/e233e5ad63a14c5cac59702f8fb9df71 2023-02-13 16:54:58,986 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/e233e5ad63a14c5cac59702f8fb9df71 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/e233e5ad63a14c5cac59702f8fb9df71 2023-02-13 16:54:58,989 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/e233e5ad63a14c5cac59702f8fb9df71, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 16:54:58,990 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 827ms, sequenceid=29, compaction requested=true 2023-02-13 16:54:58,990 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:58,990 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:58,991 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:58,992 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:58,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,992 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:58,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:58,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@361d9945[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:58,994 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:58,994 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,995 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,995 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:58,995 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:58,995 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:58,995 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,995 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:58,997 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,998 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:58,998 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:58,998 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:59,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:59,000 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:59,000 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:59,000 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:59,000 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,000 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:59,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,001 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:59,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:59,002 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,003 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,005 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,005 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:59,005 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,005 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:59,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:59,005 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:59,005 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:59,006 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:59,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:59,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 16:54:59,006 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,006 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,006 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,006 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:59,006 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:59,106 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:59,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:59,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:59,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:59,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,113 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:59,113 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:59,113 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:59,113 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:59,114 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,114 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,115 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:59,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:59,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:59,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:59,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:59,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,117 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:59,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,117 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:59,117 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:59,117 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:59,117 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:59,117 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:59,117 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:59,117 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:59,117 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:59,134 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c382d85f7c1f4ea1bc7258d901f7d4ea to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c382d85f7c1f4ea1bc7258d901f7d4ea 2023-02-13 16:54:59,142 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/826b13bfce04492db1a1aaa81e65c2c4 2023-02-13 16:54:59,146 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/826b13bfce04492db1a1aaa81e65c2c4 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/826b13bfce04492db1a1aaa81e65c2c4 2023-02-13 16:54:59,150 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/826b13bfce04492db1a1aaa81e65c2c4, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 16:54:59,150 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 33ms, sequenceid=33, compaction requested=true 2023-02-13 16:54:59,150 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:54:59,151 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:59,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:54:59,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:54:59,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:54:59,151 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:54:59,153 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,153 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:59,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:59,153 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,153 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:54:59,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:59,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:59,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:59,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:54:59,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@100e4273[Count = 0] remaining members to acquire global barrier 2023-02-13 16:54:59,155 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:54:59,155 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,156 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,156 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:54:59,156 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,156 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:54:59,156 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:54:59,156 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:54:59,158 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:54:59,158 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,158 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:59,158 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:59,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:59,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:59,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:54:59,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:54:59,160 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:54:59,160 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:54:59,160 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:54:59,160 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,160 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:54:59,161 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,161 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:59,162 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:59,162 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,163 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:54:59,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:59,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:54:59,167 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:54:59,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 16:54:59,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:59,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:54:59,166 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:59,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:59,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,167 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,167 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:54:59,167 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:54:59,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:54:59,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:54:59,267 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:54:59,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:54:59,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:54:59,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:54:59,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,274 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:54:59,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:59,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:54:59,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:54:59,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,274 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,276 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:54:59,276 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:54:59,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:54:59,276 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:54:59,276 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:54:59,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:54:59,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:54:59,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:54:59,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:54:59,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:54:59,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:54:59,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:54:59,277 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:54:59,277 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:54:59,278 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:54:59,278 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:54:59,690 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078e001061de734471aa38531c99927685 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509078e001061de734471aa38531c99927685 2023-02-13 16:55:00,097 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/176ad3a8245b4aad903582fd3a8a954d 2023-02-13 16:55:00,101 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/176ad3a8245b4aad903582fd3a8a954d as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/176ad3a8245b4aad903582fd3a8a954d 2023-02-13 16:55:00,105 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/176ad3a8245b4aad903582fd3a8a954d, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 16:55:00,106 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 828ms, sequenceid=37, compaction requested=true 2023-02-13 16:55:00,106 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:00,106 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:00,106 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:00,106 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:00,106 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,106 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:55:00,106 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:55:00,107 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:00,108 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,108 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:00,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:55:00,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4712a560[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:00,109 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:55:00,109 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,111 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,111 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:55:00,111 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:55:00,111 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:00,111 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,112 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:00,113 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:55:00,113 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:00,113 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,113 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:55:00,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:00,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:00,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:00,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:55:00,115 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:55:00,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:00,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:00,115 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,115 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:00,116 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,116 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,116 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:00,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:00,117 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,118 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,120 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:55:00,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:00,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:00,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:00,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:00,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,120 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:00,221 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:00,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:00,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:00,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:00,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,227 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:55:00,227 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:00,227 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 16:55:00,227 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 16:55:00,227 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,227 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,230 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,230 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:00,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:00,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:00,230 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:00,230 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:00,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,231 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 16:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:00,231 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 16:55:00,231 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:00,231 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 16:55:00,232 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:55:00,232 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:00,232 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:00,232 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. started... 2023-02-13 16:55:00,232 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 812c35f3f9014f1f8af17e3d2a3991cd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:55:00,644 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120547c048bae3d4ae6aea3b578bf8d3e76 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120547c048bae3d4ae6aea3b578bf8d3e76 2023-02-13 16:55:01,049 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/77a0c65f3ff6402c97a3fb896cd86885 2023-02-13 16:55:01,054 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/77a0c65f3ff6402c97a3fb896cd86885 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/77a0c65f3ff6402c97a3fb896cd86885 2023-02-13 16:55:01,059 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/77a0c65f3ff6402c97a3fb896cd86885, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 16:55:01,060 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 812c35f3f9014f1f8af17e3d2a3991cd in 828ms, sequenceid=41, compaction requested=true 2023-02-13 16:55:01,061 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:01,061 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:01,061 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:55:01,061 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:55:01,061 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,061 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 16:55:01,061 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 16:55:01,063 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,063 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:01,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:01,063 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:01,063 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:01,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:01,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:01,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 16:55:01,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50387b4b[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:01,065 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 16:55:01,065 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,066 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,066 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 16:55:01,066 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 16:55:01,066 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:01,066 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,067 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 16:55:01,068 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:01,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,068 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 16:55:01,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:01,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:01,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:01,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:01,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:01,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 16:55:01,071 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 16:55:01,071 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:01,071 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:01,071 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,071 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:01,073 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,073 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:01,073 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,074 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:01,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:01,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:01,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,075 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,078 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,078 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:01,078 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:01,078 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:01,078 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:01,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:01,078 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 16:55:01,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:55:01,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:01,078 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:01,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:01,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:01,079 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,079 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,079 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,079 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:01,079 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:01,179 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:01,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:01,181 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:01,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:01,183 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:01,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:01,184 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307901184 2023-02-13 16:55:01,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:55:01,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 16:55:01,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:01,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x6bac1159 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:01,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@342a1f04, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:01,192 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,031 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030c0e86f1723946e88ef04d009641c75a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601030c0e86f1723946e88ef04d009641c75a 2023-02-13 16:55:02,032 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:02,044 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:02,045 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53370, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:02,052 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:55:02,053 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34848, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:55:02,061 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/31455165bb514c95951027200248a002 first=Optional[r3] last=Optional[r6] 2023-02-13 16:55:02,062 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/31455165bb514c95951027200248a002}] 2023-02-13 16:55:02,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/31455165bb514c95951027200248a002 for inclusion in 812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:02,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 16:55:02,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:55:02,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:02,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/31455165bb514c95951027200248a002 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7nqf60rg5s1pgc64hrqkui87hqk01bluhp7p6h2fcsh2m0f3bh1ekfah0aarnvjq/family1/31455165bb514c95951027200248a002 2023-02-13 16:55:02,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7nqf60rg5s1pgc64hrqkui87hqk01bluhp7p6h2fcsh2m0f3bh1ekfah0aarnvjq/family1/31455165bb514c95951027200248a002 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_ 2023-02-13 16:55:02,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7nqf60rg5s1pgc64hrqkui87hqk01bluhp7p6h2fcsh2m0f3bh1ekfah0aarnvjq/family1/31455165bb514c95951027200248a002 into 812c35f3f9014f1f8af17e3d2a3991cd/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_ - updating store file list. 2023-02-13 16:55:02,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_ into 812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:02,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7nqf60rg5s1pgc64hrqkui87hqk01bluhp7p6h2fcsh2m0f3bh1ekfah0aarnvjq/family1/31455165bb514c95951027200248a002 into 812c35f3f9014f1f8af17e3d2a3991cd/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_) 2023-02-13 16:55:02,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7nqf60rg5s1pgc64hrqkui87hqk01bluhp7p6h2fcsh2m0f3bh1ekfah0aarnvjq/family1/31455165bb514c95951027200248a002 2023-02-13 16:55:02,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:55:02,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:55:02,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): 812c35f3f9014f1f8af17e3d2a3991cd/family1 is initiating minor compaction 2023-02-13 16:55:02,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5a642ad4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:02,080 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 812c35f3f9014f1f8af17e3d2a3991cd/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:02,081 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b620f31466e647d48207dfe4e6c58764, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/ebc547263d2c4f3d8e92202b192afe64, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/dc72aeab8a344d5a9930e28250477a56, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/d1b941766af24addac6349a03148b852, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/44688fbbcdcb45e7bf96661b1d6e7775, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/e233e5ad63a14c5cac59702f8fb9df71, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/826b13bfce04492db1a1aaa81e65c2c4, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/176ad3a8245b4aad903582fd3a8a954d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/77a0c65f3ff6402c97a3fb896cd86885, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp, totalSize=50.3 K 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting b620f31466e647d48207dfe4e6c58764, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting ebc547263d2c4f3d8e92202b192afe64, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting dc72aeab8a344d5a9930e28250477a56, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting d1b941766af24addac6349a03148b852, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 44688fbbcdcb45e7bf96661b1d6e7775, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting e233e5ad63a14c5cac59702f8fb9df71, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 826b13bfce04492db1a1aaa81e65c2c4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 176ad3a8245b4aad903582fd3a8a954d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 77a0c65f3ff6402c97a3fb896cd86885, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:55:02,081 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:55:02,083 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:02,084 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:02,085 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122849252fa0c3f04905b1d68c11830cfc6a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122849252fa0c3f04905b1d68c11830cfc6a 2023-02-13 16:55:02,087 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c1903d5d831e403084fa23190268262f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c1903d5d831e403084fa23190268262f 2023-02-13 16:55:02,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601016a9284431ed441a9bc37be071b8a0d26 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601016a9284431ed441a9bc37be071b8a0d26 2023-02-13 16:55:02,089 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c382d85f7c1f4ea1bc7258d901f7d4ea to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c382d85f7c1f4ea1bc7258d901f7d4ea 2023-02-13 16:55:02,089 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-13 16:55:02,089 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 16:55:02,090 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,090 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,117 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 812c35f3f9014f1f8af17e3d2a3991cd#family1#compaction#72 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:02,933 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/c4ab50a4b29746b29feb3ec7f7713011 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/c4ab50a4b29746b29feb3ec7f7713011 2023-02-13 16:55:02,937 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 812c35f3f9014f1f8af17e3d2a3991cd/family1 of 812c35f3f9014f1f8af17e3d2a3991cd into c4ab50a4b29746b29feb3ec7f7713011(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:55:02,937 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:02,937 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676307302080; duration=0sec 2023-02-13 16:55:02,937 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:02,943 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056ef17cbd9c24408d98a1566c4ace8ac4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512056ef17cbd9c24408d98a1566c4ace8ac4 2023-02-13 16:55:02,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:02,967 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/754ff02587d6420696e646e4d7723b4d first=Optional[r00] last=Optional[r2] 2023-02-13 16:55:02,968 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/754ff02587d6420696e646e4d7723b4d}] 2023-02-13 16:55:02,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/754ff02587d6420696e646e4d7723b4d for inclusion in 812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:02,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 16:55:02,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:55:02,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:02,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/754ff02587d6420696e646e4d7723b4d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vt2q6jh8jm06917uar71auns3rjmacm0tr71ebgij7flpcfdmcg2mad4iijk9hv7/family1/754ff02587d6420696e646e4d7723b4d 2023-02-13 16:55:02,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vt2q6jh8jm06917uar71auns3rjmacm0tr71ebgij7flpcfdmcg2mad4iijk9hv7/family1/754ff02587d6420696e646e4d7723b4d as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_ 2023-02-13 16:55:02,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vt2q6jh8jm06917uar71auns3rjmacm0tr71ebgij7flpcfdmcg2mad4iijk9hv7/family1/754ff02587d6420696e646e4d7723b4d into 812c35f3f9014f1f8af17e3d2a3991cd/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_ - updating store file list. 2023-02-13 16:55:02,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_ into 812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:02,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vt2q6jh8jm06917uar71auns3rjmacm0tr71ebgij7flpcfdmcg2mad4iijk9hv7/family1/754ff02587d6420696e646e4d7723b4d into 812c35f3f9014f1f8af17e3d2a3991cd/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_) 2023-02-13 16:55:02,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vt2q6jh8jm06917uar71auns3rjmacm0tr71ebgij7flpcfdmcg2mad4iijk9hv7/family1/754ff02587d6420696e646e4d7723b4d 2023-02-13 16:55:02,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:55:02,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:55:02,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): 812c35f3f9014f1f8af17e3d2a3991cd/family1 is initiating minor compaction (all files) 2023-02-13 16:55:02,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@752b0d5c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:02,983 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 812c35f3f9014f1f8af17e3d2a3991cd/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:02,984 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/7db069ed9bf94e2aaef7799d27bfefa7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/c4ab50a4b29746b29feb3ec7f7713011, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp, totalSize=388.1 K 2023-02-13 16:55:02,984 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 7db069ed9bf94e2aaef7799d27bfefa7, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842816 2023-02-13 16:55:02,984 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting c4ab50a4b29746b29feb3ec7f7713011, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242816 2023-02-13 16:55:02,985 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842816 2023-02-13 16:55:02,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:02,986 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:02,988 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130e850559942464aa885ffae688e7a54fc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130e850559942464aa885ffae688e7a54fc 2023-02-13 16:55:02,990 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201e7aee5960a5647b999344e25b7396334 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201e7aee5960a5647b999344e25b7396334 2023-02-13 16:55:02,991 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205dea04c392669458f8ad3f23d84ba53c0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205dea04c392669458f8ad3f23d84ba53c0 2023-02-13 16:55:02,991 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-13 16:55:02,991 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 16:55:02,991 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:02,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:02,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:02,993 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:02,994 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8110 sec 2023-02-13 16:55:03,002 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 812c35f3f9014f1f8af17e3d2a3991cd#family1#compaction#73 average throughput is 25.66 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:03,034 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/.tmp/family1/a7068c12eedc4ff297b9819880309e34 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/a7068c12eedc4ff297b9819880309e34 2023-02-13 16:55:03,035 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b620f31466e647d48207dfe4e6c58764, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/ebc547263d2c4f3d8e92202b192afe64, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/dc72aeab8a344d5a9930e28250477a56, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/d1b941766af24addac6349a03148b852, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/44688fbbcdcb45e7bf96661b1d6e7775, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/e233e5ad63a14c5cac59702f8fb9df71, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/826b13bfce04492db1a1aaa81e65c2c4, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/176ad3a8245b4aad903582fd3a8a954d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/77a0c65f3ff6402c97a3fb896cd86885, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_] to archive 2023-02-13 16:55:03,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:03,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b620f31466e647d48207dfe4e6c58764 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b620f31466e647d48207dfe4e6c58764 2023-02-13 16:55:03,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/ebc547263d2c4f3d8e92202b192afe64 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/ebc547263d2c4f3d8e92202b192afe64 2023-02-13 16:55:03,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/dc72aeab8a344d5a9930e28250477a56 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/dc72aeab8a344d5a9930e28250477a56 2023-02-13 16:55:03,042 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 812c35f3f9014f1f8af17e3d2a3991cd/family1 of 812c35f3f9014f1f8af17e3d2a3991cd into a7068c12eedc4ff297b9819880309e34(size=192.4 K), total size for store is 192.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:55:03,042 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:03,042 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1676307302983; duration=0sec 2023-02-13 16:55:03,042 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:03,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/d1b941766af24addac6349a03148b852 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/d1b941766af24addac6349a03148b852 2023-02-13 16:55:03,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/44688fbbcdcb45e7bf96661b1d6e7775 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/44688fbbcdcb45e7bf96661b1d6e7775 2023-02-13 16:55:03,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/e233e5ad63a14c5cac59702f8fb9df71 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/e233e5ad63a14c5cac59702f8fb9df71 2023-02-13 16:55:03,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/826b13bfce04492db1a1aaa81e65c2c4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/826b13bfce04492db1a1aaa81e65c2c4 2023-02-13 16:55:03,048 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/176ad3a8245b4aad903582fd3a8a954d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/176ad3a8245b4aad903582fd3a8a954d 2023-02-13 16:55:03,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/77a0c65f3ff6402c97a3fb896cd86885 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/77a0c65f3ff6402c97a3fb896cd86885 2023-02-13 16:55:03,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/5e90717e7ade4d778e9eb8d22cb39bc0_SeqId_44_ 2023-02-13 16:55:03,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 16:55:03,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:03,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/7db069ed9bf94e2aaef7799d27bfefa7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/c4ab50a4b29746b29feb3ec7f7713011, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_] to archive 2023-02-13 16:55:03,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:03,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/7db069ed9bf94e2aaef7799d27bfefa7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/7db069ed9bf94e2aaef7799d27bfefa7 2023-02-13 16:55:03,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/c4ab50a4b29746b29feb3ec7f7713011 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/c4ab50a4b29746b29feb3ec7f7713011 2023-02-13 16:55:03,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/b41eca907d4942628ec9b94aac6e7ed5_SeqId_47_ 2023-02-13 16:55:03,485 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 16:55:03,486 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:03,486 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 16:55:03,488 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-13 16:55:03,491 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE}] 2023-02-13 16:55:03,491 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE 2023-02-13 16:55:03,492 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:03,492 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307303492"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307303492"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307303492"}]},"ts":"1676307303492"} 2023-02-13 16:55:03,493 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:03,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 812c35f3f9014f1f8af17e3d2a3991cd, disabling compactions & flushes 2023-02-13 16:55:03,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. after waiting 0 ms 2023-02-13 16:55:03,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 16:55:03,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 812c35f3f9014f1f8af17e3d2a3991cd: Waiting for close lock at 1676307303644Running coprocessor pre-close hooks at 1676307303644Disabling compacts and flushes for region at 1676307303644Disabling writes for close at 1676307303644Writing region close event to WAL at 1676307303646 (+2 ms)Running coprocessor post-close hooks at 1676307303650 (+4 ms)Closed at 1676307303650 2023-02-13 16:55:03,650 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 812c35f3f9014f1f8af17e3d2a3991cd to self. 2023-02-13 16:55:03,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,652 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=CLOSED 2023-02-13 16:55:03,652 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307303652"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307303652"}]},"ts":"1676307303652"} 2023-02-13 16:55:03,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-13 16:55:03,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708 in 160 msec 2023-02-13 16:55:03,654 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=true 2023-02-13 16:55:03,805 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:03,805 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307303805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307303805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307303805"}]},"ts":"1676307303805"} 2023-02-13 16:55:03,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:03,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 812c35f3f9014f1f8af17e3d2a3991cd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:55:03,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:03,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,962 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,964 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:03,964 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:03,964 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 812c35f3f9014f1f8af17e3d2a3991cd columnFamilyName family1 2023-02-13 16:55:03,969 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/a7068c12eedc4ff297b9819880309e34 2023-02-13 16:55:03,969 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(310): Store=812c35f3f9014f1f8af17e3d2a3991cd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:03,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:03,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 812c35f3f9014f1f8af17e3d2a3991cd; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72692260, jitterRate=0.08319908380508423}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:55:03,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 812c35f3f9014f1f8af17e3d2a3991cd: Running coprocessor pre-open hook at 1676307303961Writing region info on filesystem at 1676307303961Initializing all the Stores at 1676307303962 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676307303962Cleaning up temporary data from old regions at 1676307303971 (+9 ms)Cleaning up detritus from prior splits at 1676307303971Running coprocessor post-open hooks at 1676307303974 (+3 ms)Region opened successfully at 1676307303974 2023-02-13 16:55:03,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., pid=63, masterSystemTime=1676307303958 2023-02-13 16:55:03,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:03,976 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:03,976 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307303976"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307303976"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307303976"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307303976"}]},"ts":"1676307303976"} 2023-02-13 16:55:03,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-13 16:55:03,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708 in 171 msec 2023-02-13 16:55:03,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-13 16:55:03,980 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE in 487 msec 2023-02-13 16:55:03,981 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-13 16:55:03,981 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 492 msec 2023-02-13 16:55:03,983 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 906 msec 2023-02-13 16:55:03,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-13 16:55:03,985 INFO [Listener at localhost/36833] compactions.TestMobCompactor(278): alter status finished 2023-02-13 16:55:03,986 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:03,987 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:03,987 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:03,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:03,988 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307903988 2023-02-13 16:55:03,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:55:03,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 16:55:03,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:03,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x15b34593 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:03,994 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@123f342b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:03,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:03,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:03,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:03,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:03,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:04,821 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283ddac9685c3346e1be81e96a7cc4a116 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511283ddac9685c3346e1be81e96a7cc4a116 2023-02-13 16:55:04,822 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:04,834 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:04,835 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57244, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:04,841 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:55:04,842 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56844, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:55:04,851 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3de6bc0c7f240ee8a4f52286de6e10e first=Optional[r05] last=Optional[r8] 2023-02-13 16:55:04,852 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3de6bc0c7f240ee8a4f52286de6e10e}] 2023-02-13 16:55:04,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3de6bc0c7f240ee8a4f52286de6e10e for inclusion in 812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:04,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 16:55:04,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:55:04,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 812c35f3f9014f1f8af17e3d2a3991cd: 2023-02-13 16:55:04,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3de6bc0c7f240ee8a4f52286de6e10e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__re4af6qkm87eot0516qhfoslicar3c0pp15ln0enq7cn9adkt6o1kurh3gcbpeto/family1/b3de6bc0c7f240ee8a4f52286de6e10e 2023-02-13 16:55:04,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__re4af6qkm87eot0516qhfoslicar3c0pp15ln0enq7cn9adkt6o1kurh3gcbpeto/family1/b3de6bc0c7f240ee8a4f52286de6e10e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/caf78f9d829e4d9c9bd4ccee20a24f03_SeqId_53_ 2023-02-13 16:55:04,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__re4af6qkm87eot0516qhfoslicar3c0pp15ln0enq7cn9adkt6o1kurh3gcbpeto/family1/b3de6bc0c7f240ee8a4f52286de6e10e into 812c35f3f9014f1f8af17e3d2a3991cd/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/caf78f9d829e4d9c9bd4ccee20a24f03_SeqId_53_ - updating store file list. 2023-02-13 16:55:04,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/caf78f9d829e4d9c9bd4ccee20a24f03_SeqId_53_ into 812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:04,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__re4af6qkm87eot0516qhfoslicar3c0pp15ln0enq7cn9adkt6o1kurh3gcbpeto/family1/b3de6bc0c7f240ee8a4f52286de6e10e into 812c35f3f9014f1f8af17e3d2a3991cd/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/caf78f9d829e4d9c9bd4ccee20a24f03_SeqId_53_) 2023-02-13 16:55:04,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__re4af6qkm87eot0516qhfoslicar3c0pp15ln0enq7cn9adkt6o1kurh3gcbpeto/family1/b3de6bc0c7f240ee8a4f52286de6e10e 2023-02-13 16:55:04,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:04,867 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:04,868 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120547c048bae3d4ae6aea3b578bf8d3e76 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120547c048bae3d4ae6aea3b578bf8d3e76 2023-02-13 16:55:04,869 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b2d70a0c1b96422b899495ccb9a01218 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b2d70a0c1b96422b899495ccb9a01218 2023-02-13 16:55:04,869 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-13 16:55:04,869 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 16:55:04,870 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:04,870 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:04,870 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:04,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:04,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:04,871 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:04,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 885 msec 2023-02-13 16:55:04,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-13 16:55:04,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:05,324 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-13 16:55:05,325 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 16:55:05,325 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-13 16:55:05,327 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-13 16:55:05,329 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE}] 2023-02-13 16:55:05,329 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE 2023-02-13 16:55:05,330 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:05,330 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307305330"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307305330"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307305330"}]},"ts":"1676307305330"} 2023-02-13 16:55:05,331 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:05,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 812c35f3f9014f1f8af17e3d2a3991cd, disabling compactions & flushes 2023-02-13 16:55:05,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. after waiting 0 ms 2023-02-13 16:55:05,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 16:55:05,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 812c35f3f9014f1f8af17e3d2a3991cd: Waiting for close lock at 1676307305483Running coprocessor pre-close hooks at 1676307305483Disabling compacts and flushes for region at 1676307305483Disabling writes for close at 1676307305483Writing region close event to WAL at 1676307305487 (+4 ms)Running coprocessor post-close hooks at 1676307305491 (+4 ms)Closed at 1676307305491 2023-02-13 16:55:05,491 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 812c35f3f9014f1f8af17e3d2a3991cd to self. 2023-02-13 16:55:05,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,493 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=CLOSED 2023-02-13 16:55:05,493 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307305493"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307305493"}]},"ts":"1676307305493"} 2023-02-13 16:55:05,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-13 16:55:05,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708 in 163 msec 2023-02-13 16:55:05,496 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=true 2023-02-13 16:55:05,646 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:05,647 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307305646"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307305646"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307305646"}]},"ts":"1676307305646"} 2023-02-13 16:55:05,648 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:05,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 812c35f3f9014f1f8af17e3d2a3991cd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:55:05,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:05,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,804 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,804 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:05,804 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1 2023-02-13 16:55:05,805 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 812c35f3f9014f1f8af17e3d2a3991cd columnFamilyName family1 2023-02-13 16:55:05,809 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/a7068c12eedc4ff297b9819880309e34 2023-02-13 16:55:05,812 DEBUG [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/family1/caf78f9d829e4d9c9bd4ccee20a24f03_SeqId_53_ 2023-02-13 16:55:05,812 INFO [StoreOpener-812c35f3f9014f1f8af17e3d2a3991cd-1] regionserver.HStore(310): Store=812c35f3f9014f1f8af17e3d2a3991cd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:05,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:55:05,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 812c35f3f9014f1f8af17e3d2a3991cd; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65823651, jitterRate=-0.019151166081428528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:55:05,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 812c35f3f9014f1f8af17e3d2a3991cd: Running coprocessor pre-open hook at 1676307305803Writing region info on filesystem at 1676307305803Initializing all the Stores at 1676307305803Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1676307305803Cleaning up temporary data from old regions at 1676307305814 (+11 ms)Cleaning up detritus from prior splits at 1676307305815 (+1 ms)Running coprocessor post-open hooks at 1676307305817 (+2 ms)Region opened successfully at 1676307305817 2023-02-13 16:55:05,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd., pid=69, masterSystemTime=1676307305800 2023-02-13 16:55:05,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:55:05,819 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=812c35f3f9014f1f8af17e3d2a3991cd, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:05,819 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676307305819"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307305819"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307305819"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307305819"}]},"ts":"1676307305819"} 2023-02-13 16:55:05,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-13 16:55:05,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 812c35f3f9014f1f8af17e3d2a3991cd, server=jenkins-hbase4.apache.org,33907,1676307219708 in 172 msec 2023-02-13 16:55:05,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-13 16:55:05,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=812c35f3f9014f1f8af17e3d2a3991cd, REOPEN/MOVE in 492 msec 2023-02-13 16:55:05,824 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 16:55:05,824 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 496 msec 2023-02-13 16:55:05,825 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 909 msec 2023-02-13 16:55:05,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-13 16:55:05,826 INFO [Listener at localhost/36833] compactions.TestMobCompactor(278): alter status finished 2023-02-13 16:55:05,827 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 16:55:05,828 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:05,829 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:05,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:05,829 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307905829 2023-02-13 16:55:05,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:55:05,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 16:55:05,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:05,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x1a4b6326 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:05,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4c1eb94e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 16:55:05,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 16:55:05,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:05,836 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 16:55:05,837 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-02-13 16:55:05,882 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=248 (was 240) - Thread LEAK? -, OpenFileDescriptor=634 (was 606) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 227), ProcessCount=168 (was 168), AvailableMemoryMB=1245 (was 1309) 2023-02-13 16:55:05,893 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=248, OpenFileDescriptor=634, MaxFileDescriptor=60000, SystemLoadAverage=177, ProcessCount=168, AvailableMemoryMB=1245 2023-02-13 16:55:05,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 16:55:05,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:05,898 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:55:05,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-13 16:55:05,899 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:55:05,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 16:55:05,900 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:05,901 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 empty. 2023-02-13 16:55:05,901 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:05,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-13 16:55:06,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 16:55:06,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 16:55:06,308 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:06,310 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => a39fa119b37d1a56ae1ca343a0487d88, NAME => 'testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:06,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 16:55:06,719 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:06,719 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing a39fa119b37d1a56ae1ca343a0487d88, disabling compactions & flushes 2023-02-13 16:55:06,719 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:06,719 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:06,719 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. after waiting 0 ms 2023-02-13 16:55:06,720 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:06,720 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:06,720 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for a39fa119b37d1a56ae1ca343a0487d88: Waiting for close lock at 1676307306719Disabling compacts and flushes for region at 1676307306719Disabling writes for close at 1676307306720 (+1 ms)Writing region close event to WAL at 1676307306720Closed at 1676307306720 2023-02-13 16:55:06,722 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:55:06,722 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307306722"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307306722"}]},"ts":"1676307306722"} 2023-02-13 16:55:06,724 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:55:06,724 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:55:06,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307306724"}]},"ts":"1676307306724"} 2023-02-13 16:55:06,725 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-13 16:55:06,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, ASSIGN}] 2023-02-13 16:55:06,729 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, ASSIGN 2023-02-13 16:55:06,729 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:06,880 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=a39fa119b37d1a56ae1ca343a0487d88, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:06,881 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307306880"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307306880"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307306880"}]},"ts":"1676307306880"} 2023-02-13 16:55:06,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure a39fa119b37d1a56ae1ca343a0487d88, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:07,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 16:55:07,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:07,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a39fa119b37d1a56ae1ca343a0487d88, NAME => 'testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:55:07,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-13 16:55:07,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-13 16:55:07,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:07,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,040 INFO [StoreOpener-a39fa119b37d1a56ae1ca343a0487d88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,041 DEBUG [StoreOpener-a39fa119b37d1a56ae1ca343a0487d88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1 2023-02-13 16:55:07,041 DEBUG [StoreOpener-a39fa119b37d1a56ae1ca343a0487d88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1 2023-02-13 16:55:07,041 INFO [StoreOpener-a39fa119b37d1a56ae1ca343a0487d88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a39fa119b37d1a56ae1ca343a0487d88 columnFamilyName f1 2023-02-13 16:55:07,042 INFO [StoreOpener-a39fa119b37d1a56ae1ca343a0487d88-1] regionserver.HStore(310): Store=a39fa119b37d1a56ae1ca343a0487d88/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:07,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:07,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:07,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a39fa119b37d1a56ae1ca343a0487d88; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69948018, jitterRate=0.04230669140815735}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:55:07,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a39fa119b37d1a56ae1ca343a0487d88: Running coprocessor pre-open hook at 1676307307039Writing region info on filesystem at 1676307307039Initializing all the Stores at 1676307307039Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307307039Cleaning up temporary data from old regions at 1676307307044 (+5 ms)Cleaning up detritus from prior splits at 1676307307044Running coprocessor post-open hooks at 1676307307048 (+4 ms)Region opened successfully at 1676307307048 2023-02-13 16:55:07,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88., pid=73, masterSystemTime=1676307307033 2023-02-13 16:55:07,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:07,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:07,050 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=a39fa119b37d1a56ae1ca343a0487d88, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:07,050 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307307050"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307307050"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307307050"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307307050"}]},"ts":"1676307307050"} 2023-02-13 16:55:07,053 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 16:55:07,053 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure a39fa119b37d1a56ae1ca343a0487d88, server=jenkins-hbase4.apache.org,33907,1676307219708 in 169 msec 2023-02-13 16:55:07,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-13 16:55:07,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, ASSIGN in 326 msec 2023-02-13 16:55:07,054 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:55:07,054 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307307054"}]},"ts":"1676307307054"} 2023-02-13 16:55:07,056 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-13 16:55:07,059 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:55:07,059 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1630 sec 2023-02-13 16:55:07,409 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57289fe2 to 127.0.0.1:64619 2023-02-13 16:55:07,409 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:07,409 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:07,409 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6bac1159 to 127.0.0.1:64619 2023-02-13 16:55:07,409 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:07,410 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a4b6326 to 127.0.0.1:64619 2023-02-13 16:55:07,410 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:07,410 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:07,410 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x15b34593 to 127.0.0.1:64619 2023-02-13 16:55:07,410 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:07,410 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:07,410 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d971b44 to 127.0.0.1:64619 2023-02-13 16:55:07,410 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:07,410 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:07,410 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0182ce51 to 127.0.0.1:64619 2023-02-13 16:55:07,410 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:08,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 16:55:08,004 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-13 16:55:08,004 DEBUG [Listener at localhost/36833] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-13 16:55:08,006 INFO [Listener at localhost/36833] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 16:55:08,009 INFO [Listener at localhost/36833] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-13 16:55:08,009 INFO [Listener at localhost/36833] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 16:55:08,010 INFO [Listener at localhost/36833] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-13 16:55:08,014 INFO [Listener at localhost/36833] regionserver.HRegion(2744): Flushing a39fa119b37d1a56ae1ca343a0487d88 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-13 16:55:08,425 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213671a686581864088a0ed8037fb52bd84 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213671a686581864088a0ed8037fb52bd84 2023-02-13 16:55:08,569 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-13 16:55:08,569 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-02-13 16:55:08,830 INFO [Listener at localhost/36833] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/.tmp/f1/3a8533c22a994ec8bf4005fc7ed3f36c 2023-02-13 16:55:08,835 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/.tmp/f1/3a8533c22a994ec8bf4005fc7ed3f36c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c 2023-02-13 16:55:08,839 INFO [Listener at localhost/36833] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c, entries=1, sequenceid=5, filesize=4.9 K 2023-02-13 16:55:08,839 INFO [Listener at localhost/36833] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for a39fa119b37d1a56ae1ca343a0487d88 in 825ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:08,840 DEBUG [Listener at localhost/36833] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-13 16:55:08,840 DEBUG [Listener at localhost/36833] regionserver.HRegion(2445): Flush status journal for a39fa119b37d1a56ae1ca343a0487d88: 2023-02-13 16:55:08,842 INFO [Listener at localhost/36833] regionserver.HRegion(2744): Flushing a39fa119b37d1a56ae1ca343a0487d88 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-13 16:55:09,182 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:55:09,253 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213a5da70d991294c149ebfcb18871a7955 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213a5da70d991294c149ebfcb18871a7955 2023-02-13 16:55:09,658 INFO [Listener at localhost/36833] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/.tmp/f1/980c5b1251b24fe194512638f522fd86 2023-02-13 16:55:09,662 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/.tmp/f1/980c5b1251b24fe194512638f522fd86 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86 2023-02-13 16:55:09,666 INFO [Listener at localhost/36833] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86, entries=1, sequenceid=9, filesize=4.9 K 2023-02-13 16:55:09,667 INFO [Listener at localhost/36833] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for a39fa119b37d1a56ae1ca343a0487d88 in 825ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:09,667 DEBUG [Listener at localhost/36833] regionserver.HRegion(2445): Flush status journal for a39fa119b37d1a56ae1ca343a0487d88: 2023-02-13 16:55:09,668 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-13 16:55:09,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 16:55:09,670 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 16:55:09,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:09,670 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307909670 2023-02-13 16:55:09,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 16:55:09,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-13 16:55:09,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-13 16:55:09,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x49043ba2 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:09,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70692f05, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:09,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testGetAfterCompaction' and column='f1' 2023-02-13 16:55:09,719 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,719 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,770 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,770 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,821 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,821 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,872 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,872 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,923 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,923 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,974 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:09,974 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,024 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,025 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,075 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,075 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,126 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,126 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,177 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,177 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,228 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,228 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,278 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,279 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,329 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,329 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,380 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,380 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,431 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,431 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,482 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,482 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,499 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213d00985ccaa6b4264a779f974eb424553 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213d00985ccaa6b4264a779f974eb424553 2023-02-13 16:55:10,499 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testGetAfterCompaction' column='f1' 2023-02-13 16:55:10,511 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:10,512 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57260, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:10,518 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:55:10,520 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56846, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:55:10,528 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/1efc6203ac04468e8774af3e8403ca2a first=Optional[r1] last=Optional[r2] 2023-02-13 16:55:10,529 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{f1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/1efc6203ac04468e8774af3e8403ca2a}] 2023-02-13 16:55:10,533 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,533 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/1efc6203ac04468e8774af3e8403ca2a for inclusion in a39fa119b37d1a56ae1ca343a0487d88/f1 2023-02-13 16:55:10,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-13 16:55:10,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:55:10,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for a39fa119b37d1a56ae1ca343a0487d88: 2023-02-13 16:55:10,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/1efc6203ac04468e8774af3e8403ca2a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testGetAfterCompaction__phood1fflfpj0uqhiutmvov8v96flsja7shqt86q9pg4l4luqpltnsbns6g8kmlg/f1/1efc6203ac04468e8774af3e8403ca2a 2023-02-13 16:55:10,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testGetAfterCompaction__phood1fflfpj0uqhiutmvov8v96flsja7shqt86q9pg4l4luqpltnsbns6g8kmlg/f1/1efc6203ac04468e8774af3e8403ca2a as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_ 2023-02-13 16:55:10,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testGetAfterCompaction__phood1fflfpj0uqhiutmvov8v96flsja7shqt86q9pg4l4luqpltnsbns6g8kmlg/f1/1efc6203ac04468e8774af3e8403ca2a into a39fa119b37d1a56ae1ca343a0487d88/f1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_ - updating store file list. 2023-02-13 16:55:10,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_ into a39fa119b37d1a56ae1ca343a0487d88/f1 2023-02-13 16:55:10,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testGetAfterCompaction__phood1fflfpj0uqhiutmvov8v96flsja7shqt86q9pg4l4luqpltnsbns6g8kmlg/f1/1efc6203ac04468e8774af3e8403ca2a into a39fa119b37d1a56ae1ca343a0487d88/f1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_) 2023-02-13 16:55:10,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testGetAfterCompaction__phood1fflfpj0uqhiutmvov8v96flsja7shqt86q9pg4l4luqpltnsbns6g8kmlg/f1/1efc6203ac04468e8774af3e8403ca2a 2023-02-13 16:55:10,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(1912): a39fa119b37d1a56ae1ca343a0487d88/f1 is initiating minor compaction 2023-02-13 16:55:10,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@31f8ef3e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:10,545 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a39fa119b37d1a56ae1ca343a0487d88/f1 in testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:10,545 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/.tmp, totalSize=10.0 K 2023-02-13 16:55:10,545 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 980c5b1251b24fe194512638f522fd86, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:55:10,545 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:55:10,547 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testGetAfterCompaction' column='f1' 2023-02-13 16:55:10,547 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:10,549 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213671a686581864088a0ed8037fb52bd84 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213671a686581864088a0ed8037fb52bd84 2023-02-13 16:55:10,550 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213a5da70d991294c149ebfcb18871a7955 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213a5da70d991294c149ebfcb18871a7955 2023-02-13 16:55:10,550 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-13 16:55:10,550 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 16:55:10,551 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testGetAfterCompaction' and column='f1' 2023-02-13 16:55:10,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-13 16:55:10,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:55:10,552 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 16:55:10,552 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): a39fa119b37d1a56ae1ca343a0487d88#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:10,553 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 883 msec 2023-02-13 16:55:10,584 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,584 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,584 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,587 DEBUG [Listener at localhost/36833] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 16:55:10,591 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57276, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 16:55:10,591 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33907] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:10,591 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-13 16:55:10,592 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-13 16:55:10,592 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33907] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-13 16:55:10,592 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33907] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. because compaction request was cancelled 2023-02-13 16:55:10,643 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,643 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,643 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,694 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,694 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,694 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,745 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,745 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,745 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,796 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,796 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,796 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,847 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,847 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,847 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,897 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,898 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,898 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,948 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,948 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,949 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:10,999 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:10,999 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,000 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,050 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,050 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,050 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,101 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,101 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,101 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,152 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,152 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,152 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,203 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,203 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,203 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,254 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,254 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,254 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,305 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,305 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,305 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,356 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,356 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,356 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,365 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/.tmp/f1/e131d7ae520c49c384855b8776b9ceb6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/e131d7ae520c49c384855b8776b9ceb6 2023-02-13 16:55:11,369 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in a39fa119b37d1a56ae1ca343a0487d88/f1 of a39fa119b37d1a56ae1ca343a0487d88 into e131d7ae520c49c384855b8776b9ceb6(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:55:11,370 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a39fa119b37d1a56ae1ca343a0487d88: 2023-02-13 16:55:11,370 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1676307310544; duration=0sec 2023-02-13 16:55:11,370 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:11,406 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,407 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307309658; access_time=1676307308844; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,407 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/e131d7ae520c49c384855b8776b9ceb6; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676307311361; access_time=1676307310550; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,407 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676307310495; access_time=1676307309687; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 16:55:11,419 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_] to archive 2023-02-13 16:55:11,420 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:11,421 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/980c5b1251b24fe194512638f522fd86 2023-02-13 16:55:11,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/ed69b230c9b24f8f9bf37d33ecaadef3_SeqId_12_ 2023-02-13 16:55:11,457 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676307308830; access_time=1676307308016; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,457 INFO [Listener at localhost/36833] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/e131d7ae520c49c384855b8776b9ceb6; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676307311361; access_time=1676307310550; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 16:55:11,510 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 16:55:11,571 INFO [Listener at localhost/36833] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-13 16:55:11,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-13 16:55:11,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,583 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307311583"}]},"ts":"1676307311583"} 2023-02-13 16:55:11,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 16:55:11,584 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-13 16:55:11,590 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-13 16:55:11,591 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, UNASSIGN}] 2023-02-13 16:55:11,592 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, UNASSIGN 2023-02-13 16:55:11,593 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=a39fa119b37d1a56ae1ca343a0487d88, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:11,593 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307311593"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307311593"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307311593"}]},"ts":"1676307311593"} 2023-02-13 16:55:11,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure a39fa119b37d1a56ae1ca343a0487d88, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:11,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 16:55:11,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:11,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a39fa119b37d1a56ae1ca343a0487d88, disabling compactions & flushes 2023-02-13 16:55:11,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:11,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:11,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. after waiting 0 ms 2023-02-13 16:55:11,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:11,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-13 16:55:11,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-13 16:55:11,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88. 2023-02-13 16:55:11,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a39fa119b37d1a56ae1ca343a0487d88: Waiting for close lock at 1676307311746Running coprocessor pre-close hooks at 1676307311746Disabling compacts and flushes for region at 1676307311746Disabling writes for close at 1676307311746Writing region close event to WAL at 1676307311747 (+1 ms)Running coprocessor post-close hooks at 1676307311750 (+3 ms)Closed at 1676307311751 (+1 ms) 2023-02-13 16:55:11,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:11,753 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=a39fa119b37d1a56ae1ca343a0487d88, regionState=CLOSED 2023-02-13 16:55:11,753 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676307311753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307311753"}]},"ts":"1676307311753"} 2023-02-13 16:55:11,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-13 16:55:11,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure a39fa119b37d1a56ae1ca343a0487d88, server=jenkins-hbase4.apache.org,33907,1676307219708 in 160 msec 2023-02-13 16:55:11,756 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-13 16:55:11,756 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=a39fa119b37d1a56ae1ca343a0487d88, UNASSIGN in 164 msec 2023-02-13 16:55:11,756 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307311756"}]},"ts":"1676307311756"} 2023-02-13 16:55:11,757 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-13 16:55:11,758 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-13 16:55:11,760 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 183 msec 2023-02-13 16:55:11,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 16:55:11,886 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-13 16:55:11,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-13 16:55:11,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,897 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-13 16:55:11,901 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:11,903 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1, FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/recovered.edits] 2023-02-13 16:55:11,906 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/3a8533c22a994ec8bf4005fc7ed3f36c 2023-02-13 16:55:11,907 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/e131d7ae520c49c384855b8776b9ceb6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/f1/e131d7ae520c49c384855b8776b9ceb6 2023-02-13 16:55:11,910 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/recovered.edits/15.seqid to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88/recovered.edits/15.seqid 2023-02-13 16:55:11,910 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testGetAfterCompaction/a39fa119b37d1a56ae1ca343a0487d88 2023-02-13 16:55:11,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-13 16:55:11,911 DEBUG [PEWorker-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-13 16:55:11,911 DEBUG [PEWorker-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-13 16:55:11,914 DEBUG [PEWorker-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213d00985ccaa6b4264a779f974eb424553 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213d00985ccaa6b4264a779f974eb424553 2023-02-13 16:55:11,914 DEBUG [PEWorker-1] backup.HFileArchiver(596): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-13 16:55:11,917 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,938 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-13 16:55:11,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-13 16:55:11,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-13 16:55:11,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307311942"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:11,944 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 16:55:11,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a39fa119b37d1a56ae1ca343a0487d88, NAME => 'testGetAfterCompaction,,1676307305896.a39fa119b37d1a56ae1ca343a0487d88.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 16:55:11,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-13 16:55:11,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676307311944"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:11,945 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-13 16:55:11,947 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 16:55:11,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 56 msec 2023-02-13 16:55:12,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-13 16:55:12,000 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-13 16:55:12,013 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=234 (was 248), OpenFileDescriptor=597 (was 634), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=158 (was 177), ProcessCount=168 (was 168), AvailableMemoryMB=1213 (was 1245) 2023-02-13 16:55:12,023 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=234, OpenFileDescriptor=597, MaxFileDescriptor=60000, SystemLoadAverage=158, ProcessCount=168, AvailableMemoryMB=1212 2023-02-13 16:55:12,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 16:55:12,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-13 16:55:12,027 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:55:12,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-13 16:55:12,027 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:55:12,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:55:12,029 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:12,029 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:12,029 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:12,030 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 empty. 2023-02-13 16:55:12,030 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 empty. 2023-02-13 16:55:12,030 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 empty. 2023-02-13 16:55:12,030 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:12,030 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:12,030 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:12,030 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-13 16:55:12,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:55:12,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:55:12,438 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:12,439 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => cf586ed2ad20940a6fec159be48a9844, NAME => 'testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:12,440 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => a5218e16a3f58aacc78158f34c1f6d13, NAME => 'testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:12,440 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 039a7376bb8abccd044c10092dd55a94, NAME => 'testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:12,447 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:12,447 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing cf586ed2ad20940a6fec159be48a9844, disabling compactions & flushes 2023-02-13 16:55:12,447 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:12,447 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:12,447 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. after waiting 0 ms 2023-02-13 16:55:12,447 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:12,447 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:12,447 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for cf586ed2ad20940a6fec159be48a9844: Waiting for close lock at 1676307312447Disabling compacts and flushes for region at 1676307312447Disabling writes for close at 1676307312447Writing region close event to WAL at 1676307312447Closed at 1676307312447 2023-02-13 16:55:12,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:55:12,850 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:12,850 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 039a7376bb8abccd044c10092dd55a94, disabling compactions & flushes 2023-02-13 16:55:12,850 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:12,850 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:12,850 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. after waiting 0 ms 2023-02-13 16:55:12,850 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:12,850 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:12,850 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 039a7376bb8abccd044c10092dd55a94: Waiting for close lock at 1676307312850Disabling compacts and flushes for region at 1676307312850Disabling writes for close at 1676307312850Writing region close event to WAL at 1676307312850Closed at 1676307312850 2023-02-13 16:55:12,852 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:12,852 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing a5218e16a3f58aacc78158f34c1f6d13, disabling compactions & flushes 2023-02-13 16:55:12,852 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:12,852 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:12,852 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. after waiting 0 ms 2023-02-13 16:55:12,852 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:12,852 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:12,852 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for a5218e16a3f58aacc78158f34c1f6d13: Waiting for close lock at 1676307312852Disabling compacts and flushes for region at 1676307312852Disabling writes for close at 1676307312852Writing region close event to WAL at 1676307312852Closed at 1676307312852 2023-02-13 16:55:12,854 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:55:12,855 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307312855"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307312855"}]},"ts":"1676307312855"} 2023-02-13 16:55:12,855 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307312855"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307312855"}]},"ts":"1676307312855"} 2023-02-13 16:55:12,855 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307312855"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307312855"}]},"ts":"1676307312855"} 2023-02-13 16:55:12,857 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 16:55:12,858 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:55:12,858 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307312858"}]},"ts":"1676307312858"} 2023-02-13 16:55:12,859 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-13 16:55:12,864 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN}] 2023-02-13 16:55:12,866 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN 2023-02-13 16:55:12,866 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN 2023-02-13 16:55:12,866 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN 2023-02-13 16:55:12,866 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:12,866 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:12,866 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:13,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=cf586ed2ad20940a6fec159be48a9844, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:13,018 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=039a7376bb8abccd044c10092dd55a94, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:13,018 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=a5218e16a3f58aacc78158f34c1f6d13, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:13,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307313018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307313018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307313018"}]},"ts":"1676307313018"} 2023-02-13 16:55:13,018 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307313018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307313018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307313018"}]},"ts":"1676307313018"} 2023-02-13 16:55:13,018 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307313018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307313018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307313018"}]},"ts":"1676307313018"} 2023-02-13 16:55:13,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure cf586ed2ad20940a6fec159be48a9844, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:13,020 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 039a7376bb8abccd044c10092dd55a94, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:13,022 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure a5218e16a3f58aacc78158f34c1f6d13, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:13,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:55:13,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:13,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a5218e16a3f58aacc78158f34c1f6d13, NAME => 'testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 16:55:13,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:13,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,176 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,178 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:13,178 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:13,178 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5218e16a3f58aacc78158f34c1f6d13 columnFamilyName family1 2023-02-13 16:55:13,180 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(310): Store=a5218e16a3f58aacc78158f34c1f6d13/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:13,180 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,181 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2 2023-02-13 16:55:13,181 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2 2023-02-13 16:55:13,182 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5218e16a3f58aacc78158f34c1f6d13 columnFamilyName family2 2023-02-13 16:55:13,183 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(310): Store=a5218e16a3f58aacc78158f34c1f6d13/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:13,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:13,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:13,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:13,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a5218e16a3f58aacc78158f34c1f6d13; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66587853, jitterRate=-0.007763668894767761}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:13,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a5218e16a3f58aacc78158f34c1f6d13: Running coprocessor pre-open hook at 1676307313175Writing region info on filesystem at 1676307313175Initializing all the Stores at 1676307313176 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307313176Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307313176Cleaning up temporary data from old regions at 1676307313185 (+9 ms)Cleaning up detritus from prior splits at 1676307313185Running coprocessor post-open hooks at 1676307313189 (+4 ms)Region opened successfully at 1676307313189 2023-02-13 16:55:13,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13., pid=85, masterSystemTime=1676307313172 2023-02-13 16:55:13,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:13,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:13,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:13,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cf586ed2ad20940a6fec159be48a9844, NAME => 'testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 16:55:13,191 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=a5218e16a3f58aacc78158f34c1f6d13, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:13,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:13,192 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307313191"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307313191"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307313191"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307313191"}]},"ts":"1676307313191"} 2023-02-13 16:55:13,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,193 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,194 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:13,194 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:13,194 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-02-13 16:55:13,194 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cf586ed2ad20940a6fec159be48a9844 columnFamilyName family1 2023-02-13 16:55:13,194 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure a5218e16a3f58aacc78158f34c1f6d13, server=jenkins-hbase4.apache.org,33907,1676307219708 in 172 msec 2023-02-13 16:55:13,195 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(310): Store=cf586ed2ad20940a6fec159be48a9844/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:13,195 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN in 331 msec 2023-02-13 16:55:13,195 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,196 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2 2023-02-13 16:55:13,196 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2 2023-02-13 16:55:13,197 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cf586ed2ad20940a6fec159be48a9844 columnFamilyName family2 2023-02-13 16:55:13,197 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(310): Store=cf586ed2ad20940a6fec159be48a9844/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:13,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:13,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:13,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:13,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cf586ed2ad20940a6fec159be48a9844; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59747562, jitterRate=-0.10969194769859314}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:13,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cf586ed2ad20940a6fec159be48a9844: Running coprocessor pre-open hook at 1676307313192Writing region info on filesystem at 1676307313192Initializing all the Stores at 1676307313192Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307313192Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307313193 (+1 ms)Cleaning up temporary data from old regions at 1676307313199 (+6 ms)Cleaning up detritus from prior splits at 1676307313199Running coprocessor post-open hooks at 1676307313202 (+3 ms)Region opened successfully at 1676307313202 2023-02-13 16:55:13,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844., pid=83, masterSystemTime=1676307313172 2023-02-13 16:55:13,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:13,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:13,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:13,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 039a7376bb8abccd044c10092dd55a94, NAME => 'testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 16:55:13,204 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=cf586ed2ad20940a6fec159be48a9844, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:13,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:13,204 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307313204"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307313204"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307313204"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307313204"}]},"ts":"1676307313204"} 2023-02-13 16:55:13,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,205 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,206 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:13,207 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:13,207 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-02-13 16:55:13,207 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure cf586ed2ad20940a6fec159be48a9844, server=jenkins-hbase4.apache.org,33907,1676307219708 in 186 msec 2023-02-13 16:55:13,207 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 039a7376bb8abccd044c10092dd55a94 columnFamilyName family1 2023-02-13 16:55:13,208 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(310): Store=039a7376bb8abccd044c10092dd55a94/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:13,208 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN in 343 msec 2023-02-13 16:55:13,208 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,209 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2 2023-02-13 16:55:13,209 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2 2023-02-13 16:55:13,209 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 039a7376bb8abccd044c10092dd55a94 columnFamilyName family2 2023-02-13 16:55:13,210 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(310): Store=039a7376bb8abccd044c10092dd55a94/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:13,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:13,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:13,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:13,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 039a7376bb8abccd044c10092dd55a94; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68986051, jitterRate=0.0279722660779953}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:13,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 039a7376bb8abccd044c10092dd55a94: Running coprocessor pre-open hook at 1676307313205Writing region info on filesystem at 1676307313205Initializing all the Stores at 1676307313205Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307313205Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307313205Cleaning up temporary data from old regions at 1676307313212 (+7 ms)Cleaning up detritus from prior splits at 1676307313212Running coprocessor post-open hooks at 1676307313216 (+4 ms)Region opened successfully at 1676307313216 2023-02-13 16:55:13,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94., pid=84, masterSystemTime=1676307313172 2023-02-13 16:55:13,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:13,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:13,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=039a7376bb8abccd044c10092dd55a94, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:13,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307313218"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307313218"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307313218"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307313218"}]},"ts":"1676307313218"} 2023-02-13 16:55:13,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-02-13 16:55:13,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 039a7376bb8abccd044c10092dd55a94, server=jenkins-hbase4.apache.org,33907,1676307219708 in 199 msec 2023-02-13 16:55:13,221 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-13 16:55:13,221 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN in 357 msec 2023-02-13 16:55:13,222 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:55:13,222 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307313222"}]},"ts":"1676307313222"} 2023-02-13 16:55:13,223 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-13 16:55:13,225 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:55:13,226 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2000 sec 2023-02-13 16:55:14,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 16:55:14,132 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-13 16:55:14,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:14,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:14,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:14,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:14,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 16:55:14,145 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 16:55:14,145 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:14,145 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 16:55:14,145 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 16:55:14,145 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:14,145 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:14,147 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:14,147 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:14,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:14,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:14,147 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:14,147 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:14,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:14,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:14,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:14,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:14,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 16:55:14,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 16:55:14,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 16:55:14,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:14,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 16:55:14,148 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:14,148 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. started... 2023-02-13 16:55:14,148 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:14,148 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:14,149 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:14,148 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. started... 2023-02-13 16:55:14,149 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:14,149 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. started... 2023-02-13 16:55:14,149 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 039a7376bb8abccd044c10092dd55a94 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:55:14,149 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing cf586ed2ad20940a6fec159be48a9844 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:55:14,150 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a5218e16a3f58aacc78158f34c1f6d13 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:55:14,171 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e 2023-02-13 16:55:14,567 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 2023-02-13 16:55:14,569 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d 2023-02-13 16:55:14,576 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/018e35dc694942c0bfd1e84103d963a8 2023-02-13 16:55:14,972 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/9eb60b70ae754b51a22135f04efc35d6 2023-02-13 16:55:14,974 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/4659535d245c4680bb0a591c028508ad 2023-02-13 16:55:14,987 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302139ef25c70d9d74a318fea3ef635ab895b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302139ef25c70d9d74a318fea3ef635ab895b 2023-02-13 16:55:15,387 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e69eebf3bf1f423785cd144cfb6762e9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e69eebf3bf1f423785cd144cfb6762e9 2023-02-13 16:55:15,387 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302135b8c91ec7211473189c37abfa5d48e13 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302135b8c91ec7211473189c37abfa5d48e13 2023-02-13 16:55:15,392 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/0bbeac208b0c42539fa029aff69fc201 2023-02-13 16:55:15,394 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/168873072c92404289dc5f8f0e25e5d3 2023-02-13 16:55:15,394 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/0a88ddb76fe5482797752b8c11c57fb0 2023-02-13 16:55:15,396 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/018e35dc694942c0bfd1e84103d963a8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8 2023-02-13 16:55:15,398 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/9eb60b70ae754b51a22135f04efc35d6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6 2023-02-13 16:55:15,398 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/4659535d245c4680bb0a591c028508ad as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad 2023-02-13 16:55:15,400 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 16:55:15,401 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/0bbeac208b0c42539fa029aff69fc201 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201 2023-02-13 16:55:15,402 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 16:55:15,403 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/168873072c92404289dc5f8f0e25e5d3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3 2023-02-13 16:55:15,403 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 16:55:15,404 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/0a88ddb76fe5482797752b8c11c57fb0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0 2023-02-13 16:55:15,406 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:55:15,407 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:55:15,407 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for cf586ed2ad20940a6fec159be48a9844 in 1258ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:15,407 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 16:55:15,408 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:55:15,408 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:15,408 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:15,408 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:15,408 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 039a7376bb8abccd044c10092dd55a94 in 1259ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:15,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:15,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:15,409 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a5218e16a3f58aacc78158f34c1f6d13 in 1259ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:15,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:15,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 16:55:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 16:55:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,412 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:15,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 16:55:15,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ab517b8[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:15,413 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 16:55:15,413 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,416 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,417 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 16:55:15,417 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 16:55:15,417 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:15,417 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,417 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:15,418 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 16:55:15,418 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:15,418 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,418 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 16:55:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:15,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 16:55:15,420 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 16:55:15,420 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:15,420 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:15,420 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 16:55:15,420 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:15,422 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,422 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:15,422 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:15,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:15,423 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:15,424 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,426 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:15,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:15,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 16:55:15,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:15,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:15,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:15,427 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:15,427 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:15,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:15,527 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:15,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:15,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:15,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:15,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:15,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:15,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:15,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 16:55:15,534 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 16:55:15,534 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:15,534 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 16:55:15,534 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 16:55:15,534 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,534 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:15,537 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,537 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:15,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:15,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:15,537 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:15,537 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:15,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:15,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:15,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:15,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:15,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 16:55:15,539 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 16:55:15,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 16:55:15,539 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 16:55:15,539 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:15,539 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 16:55:15,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:15,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. started... 2023-02-13 16:55:15,539 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:15,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:15,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:15,539 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:15,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. started... 2023-02-13 16:55:15,539 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 039a7376bb8abccd044c10092dd55a94 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:55:15,540 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a5218e16a3f58aacc78158f34c1f6d13 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:55:15,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. started... 2023-02-13 16:55:15,540 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing cf586ed2ad20940a6fec159be48a9844 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:55:15,957 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe 2023-02-13 16:55:15,961 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 2023-02-13 16:55:15,967 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe 2023-02-13 16:55:16,363 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/adc5d5b07a8c4a119fa1aa17c983d6b9 2023-02-13 16:55:16,365 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/9a900594a63745a1a09e0a6288d70be3 2023-02-13 16:55:16,371 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/299fc7a5672047c697fc48979fca6ca1 2023-02-13 16:55:16,378 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302131fca19cf1513448a80061b9d20df2148 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302131fca19cf1513448a80061b9d20df2148 2023-02-13 16:55:16,777 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213d4d35bcf268a4258b2601beba113a23e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213d4d35bcf268a4258b2601beba113a23e 2023-02-13 16:55:16,788 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021335c7444662c4437fb47151ede48018ff to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021335c7444662c4437fb47151ede48018ff 2023-02-13 16:55:16,789 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/22a5bb35b55349b29dfcda07f6d868d9 2023-02-13 16:55:16,793 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/9a900594a63745a1a09e0a6288d70be3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3 2023-02-13 16:55:16,797 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 16:55:16,797 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/22a5bb35b55349b29dfcda07f6d868d9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9 2023-02-13 16:55:16,800 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:55:16,801 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a5218e16a3f58aacc78158f34c1f6d13 in 1262ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:16,801 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:16,801 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:16,801 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:17,182 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/87e141fd145f4df4a134a3ed29c37476 2023-02-13 16:55:17,186 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/adc5d5b07a8c4a119fa1aa17c983d6b9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9 2023-02-13 16:55:17,189 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 16:55:17,190 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/87e141fd145f4df4a134a3ed29c37476 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476 2023-02-13 16:55:17,193 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/f6ea0d3ea6194f0690b02fe9f4eee202 2023-02-13 16:55:17,194 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:55:17,195 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 039a7376bb8abccd044c10092dd55a94 in 1656ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:17,195 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:17,195 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:17,198 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/299fc7a5672047c697fc48979fca6ca1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1 2023-02-13 16:55:17,202 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 16:55:17,202 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/f6ea0d3ea6194f0690b02fe9f4eee202 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202 2023-02-13 16:55:17,207 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:55:17,207 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for cf586ed2ad20940a6fec159be48a9844 in 1667ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:17,207 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:17,207 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:17,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:17,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:17,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,208 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 16:55:17,208 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 16:55:17,209 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,209 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:17,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,210 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:17,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 16:55:17,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@684ad147[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:17,211 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 16:55:17,211 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,212 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 16:55:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 16:55:17,212 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:17,212 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,212 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:17,213 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,213 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 16:55:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:17,214 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:17,214 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 16:55:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:17,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:17,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:17,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:17,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 16:55:17,216 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 16:55:17,216 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:17,216 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:17,216 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 16:55:17,216 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:17,218 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,218 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,218 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:17,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:17,218 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:17,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:17,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:17,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:17,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:17,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:17,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:17,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:17,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 16:55:17,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:17,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:17,221 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:17,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:17,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:17,222 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,222 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,222 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:17,222 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,222 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:17,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:17,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:17,222 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:17,252 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:55:17,322 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:17,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:17,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:17,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:17,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:17,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:17,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:17,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 16:55:17,329 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 16:55:17,329 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:17,329 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 16:55:17,329 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 16:55:17,329 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,329 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:17,331 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,331 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:17,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:17,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:17,331 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:17,331 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:17,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:17,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:17,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:17,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:17,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 16:55:17,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 16:55:17,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 16:55:17,332 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 16:55:17,332 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:17,332 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 16:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:17,333 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. started... 2023-02-13 16:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. started... 2023-02-13 16:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:17,333 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 039a7376bb8abccd044c10092dd55a94 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:55:17,333 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:17,333 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a5218e16a3f58aacc78158f34c1f6d13 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. started... 2023-02-13 16:55:17,333 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing cf586ed2ad20940a6fec159be48a9844 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:55:17,753 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df 2023-02-13 16:55:17,754 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 2023-02-13 16:55:17,754 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 2023-02-13 16:55:18,168 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/b8332ec2185f4bf29c4897a09af546fb 2023-02-13 16:55:18,168 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/9d97f86755984e3f8da69d2eb9150580 2023-02-13 16:55:18,168 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/8b5754ecd7b64c539ea5bf7491cbf4b0 2023-02-13 16:55:18,569 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-13 16:55:18,587 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021359add7626ed24e708979796f06ec3797 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021359add7626ed24e708979796f06ec3797 2023-02-13 16:55:18,587 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302132ade7814c9294d99a942fe57260cab93 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302132ade7814c9294d99a942fe57260cab93 2023-02-13 16:55:18,587 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302136046d46688ba416ba3aa4485e555c6e3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302136046d46688ba416ba3aa4485e555c6e3 2023-02-13 16:55:18,992 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/c3a279f58e664601bf85861d9822473b 2023-02-13 16:55:18,992 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/542e06fb7daf47e48c2be60727e2cf0c 2023-02-13 16:55:18,992 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/6a1c1bf3d4c34889a706fff2bac1051c 2023-02-13 16:55:18,997 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/9d97f86755984e3f8da69d2eb9150580 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580 2023-02-13 16:55:18,997 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/8b5754ecd7b64c539ea5bf7491cbf4b0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0 2023-02-13 16:55:19,001 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 16:55:19,001 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 16:55:19,001 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/b8332ec2185f4bf29c4897a09af546fb as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb 2023-02-13 16:55:19,001 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/542e06fb7daf47e48c2be60727e2cf0c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c 2023-02-13 16:55:19,001 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/6a1c1bf3d4c34889a706fff2bac1051c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c 2023-02-13 16:55:19,005 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 16:55:19,005 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:55:19,005 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:55:19,006 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for cf586ed2ad20940a6fec159be48a9844 in 1673ms, sequenceid=13, compaction requested=true 2023-02-13 16:55:19,006 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a5218e16a3f58aacc78158f34c1f6d13 in 1673ms, sequenceid=13, compaction requested=true 2023-02-13 16:55:19,006 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:19,006 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:19,006 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:19,006 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:19,007 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/c3a279f58e664601bf85861d9822473b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b 2023-02-13 16:55:19,011 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:55:19,011 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 039a7376bb8abccd044c10092dd55a94 in 1678ms, sequenceid=13, compaction requested=true 2023-02-13 16:55:19,011 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:19,012 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:19,012 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:19,012 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:19,012 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,012 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 16:55:19,012 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 16:55:19,013 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,013 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:19,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:19,014 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,014 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:19,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:19,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:19,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:19,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:19,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 16:55:19,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3aa1508b[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:19,015 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 16:55:19,015 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,017 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,017 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,017 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 16:55:19,017 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 16:55:19,017 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:19,017 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:19,021 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,021 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:19,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:19,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:19,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 16:55:19,023 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 16:55:19,023 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:19,024 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:19,024 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 16:55:19,024 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:19,025 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,025 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,025 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:19,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:19,026 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:19,026 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:19,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:19,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:19,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:19,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:19,029 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:19,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:19,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:19,029 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:19,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:19,030 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:19,130 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:19,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:19,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:19,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:19,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:19,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:19,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:19,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 16:55:19,136 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 16:55:19,136 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:19,136 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 16:55:19,136 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 16:55:19,136 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,136 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:19,137 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,137 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:19,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:19,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:19,138 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:19,138 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:19,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:19,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:19,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:19,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:19,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 16:55:19,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 16:55:19,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 16:55:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 16:55:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 16:55:19,139 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:19,139 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. started... 2023-02-13 16:55:19,139 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:19,139 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. started... 2023-02-13 16:55:19,139 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:19,139 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 039a7376bb8abccd044c10092dd55a94 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:55:19,139 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing cf586ed2ad20940a6fec159be48a9844 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:55:19,139 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. started... 2023-02-13 16:55:19,139 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a5218e16a3f58aacc78158f34c1f6d13 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:55:19,161 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e 2023-02-13 16:55:19,557 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 2023-02-13 16:55:19,558 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a 2023-02-13 16:55:19,567 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/79c464dcbeae4ad8b0567e964b3957c6 2023-02-13 16:55:19,568 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/78ee9a9d7b274418a710aaa7b139efbe 2023-02-13 16:55:19,962 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/438d44af37cf495b92d1e9d2a48f2d6a 2023-02-13 16:55:19,985 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213e3fc9a6bdedc411f87fbe08abbb0d581 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213e3fc9a6bdedc411f87fbe08abbb0d581 2023-02-13 16:55:19,985 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302138e9b9b6df18c47ebbe982f05b25a2079 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302138e9b9b6df18c47ebbe982f05b25a2079 2023-02-13 16:55:20,376 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213f91b8ea431d749d9921fd54813e91e39 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213f91b8ea431d749d9921fd54813e91e39 2023-02-13 16:55:20,390 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/9b80ab619d004318a0c72ad5f64a2dd7 2023-02-13 16:55:20,390 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/dc077abab99a4109b4db3728d7e0b2ae 2023-02-13 16:55:20,393 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/78ee9a9d7b274418a710aaa7b139efbe as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe 2023-02-13 16:55:20,393 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/79c464dcbeae4ad8b0567e964b3957c6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6 2023-02-13 16:55:20,396 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 16:55:20,397 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 16:55:20,397 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/9b80ab619d004318a0c72ad5f64a2dd7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7 2023-02-13 16:55:20,397 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/dc077abab99a4109b4db3728d7e0b2ae as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae 2023-02-13 16:55:20,400 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:55:20,400 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:55:20,401 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 039a7376bb8abccd044c10092dd55a94 in 1262ms, sequenceid=17, compaction requested=true 2023-02-13 16:55:20,401 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for cf586ed2ad20940a6fec159be48a9844 in 1262ms, sequenceid=17, compaction requested=true 2023-02-13 16:55:20,401 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:20,401 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:20,401 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:20,401 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:20,402 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:20,402 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:20,780 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/c235dcb64d0940f2be7603e3a595f6f7 2023-02-13 16:55:20,785 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/438d44af37cf495b92d1e9d2a48f2d6a as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a 2023-02-13 16:55:20,788 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 16:55:20,789 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/c235dcb64d0940f2be7603e3a595f6f7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7 2023-02-13 16:55:20,793 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:55:20,793 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a5218e16a3f58aacc78158f34c1f6d13 in 1654ms, sequenceid=17, compaction requested=true 2023-02-13 16:55:20,793 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:20,793 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:20,793 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:20,793 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:20,793 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,794 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 16:55:20,794 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 16:55:20,795 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,795 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:20,795 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:20,795 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 16:55:20,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29d096b7[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:20,797 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 16:55:20,797 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,798 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 16:55:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 16:55:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:20,798 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,798 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:20,800 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,800 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 16:55:20,800 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,800 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 16:55:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:20,802 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 16:55:20,802 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:20,802 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:20,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 16:55:20,802 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 16:55:20,802 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:20,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:20,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:20,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,803 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:20,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:20,805 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:20,806 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:20,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 16:55:20,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:20,809 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:20,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:20,809 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:20,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:20,909 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:20,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:20,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:20,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676307320920 (current time:1676307320920). 2023-02-13 16:55:20,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-13 16:55:20,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] zookeeper.ReadOnlyZKClient(139): Connect 0x77a306a1 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:20,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1a5aca4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:20,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:20,934 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56386, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:20,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77a306a1 to 127.0.0.1:64619 2023-02-13 16:55:20,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:20,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-13 16:55:20,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:20,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 16:55:20,944 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 16:55:20,945 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676307920945 2023-02-13 16:55:20,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:20,946 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1676307320910 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-13 16:55:20,947 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 16:55:20,948 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 16:55:20,948 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-02-13 16:55:20,948 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 16:55:20,949 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1676307920949 2023-02-13 16:55:20,950 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:20,951 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-13 16:55:21,051 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:21,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:21,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:21,055 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-13 16:55:21,255 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:21,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:21,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:21,256 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-13 16:55:21,364 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1676307320910 2023-02-13 16:55:21,365 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1676307320910' 2023-02-13 16:55:21,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-13 16:55:21,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1676307320910' starting 'acquire' 2023-02-13 16:55:21,365 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1676307320910', kicking off acquire phase on members. 2023-02-13 16:55:21,366 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,366 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1676307320910 2023-02-13 16:55:21,367 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,367 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-13 16:55:21,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-13 16:55:21,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 16:55:21,367 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,367 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:21,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1676307320910 2023-02-13 16:55:21,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-13 16:55:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1676307320910 2023-02-13 16:55:21,368 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1676307320910 from table testCompactionWithHFileLink type FLUSH 2023-02-13 16:55:21,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1676307320910 2023-02-13 16:55:21,370 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1676307320910' with timeout 300000ms 2023-02-13 16:55:21,370 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-13 16:55:21,370 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1676307320910' starting 'acquire' stage 2023-02-13 16:55:21,370 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1676307320910' locally acquired 2023-02-13 16:55:21,370 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (snaptb-1676307320910) in zk 2023-02-13 16:55:21,372 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,372 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:21,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 16:55:21,372 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,372 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1676307320910' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:21,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676307320910 2023-02-13 16:55:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:21,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'snaptb-1676307320910' on coordinator 2023-02-13 16:55:21,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b7a5ce2[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:21,380 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1676307320910' starting 'in-barrier' execution. 2023-02-13 16:55:21,380 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,382 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,382 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1676307320910' received 'reached' from coordinator. 2023-02-13 16:55:21,382 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,382 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:21,383 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-13 16:55:21,383 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. started... 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. started... 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. started... 2023-02-13 16:55:21,383 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:21,384 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:21,384 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:21,384 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.' region-info for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,384 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.' region-info for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,384 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.' region-info for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 16:55:21,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 16:55:21,390 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 16:55:21,538 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:21,538 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-13 16:55:21,538 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49043ba2 to 127.0.0.1:64619 2023-02-13 16:55:21,538 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:21,538 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a] hfiles 2023-02-13 16:55:21,538 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6] hfiles 2023-02-13 16:55:21,538 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe] hfiles 2023-02-13 16:55:21,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,539 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae] hfiles 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7] hfiles 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7] hfiles 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,544 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,544 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,543 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,544 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,544 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,544 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7 for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,556 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:21,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:21,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:21,557 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-13 16:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. completed. 2023-02-13 16:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. completed. 2023-02-13 16:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. completed. 2023-02-13 16:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1676307320910' locally completed 2023-02-13 16:55:21,951 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1676307320910' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:21,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1676307320910' has notified controller of completion 2023-02-13 16:55:21,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:21,953 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,953 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1676307320910' completed. 2023-02-13 16:55:21,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:21,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 16:55:21,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:21,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676307320910 2023-02-13 16:55:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676307320910 2023-02-13 16:55:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1676307320910' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:21,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'snaptb-1676307320910', counting down latch. Waiting for 0 more 2023-02-13 16:55:21,956 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1676307320910' execution completed 2023-02-13 16:55:21,956 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:21,956 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:21,956 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1676307320910 2023-02-13 16:55:21,956 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1676307320910including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-13 16:55:21,957 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,957 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,957 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-13 16:55:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 16:55:21,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-13 16:55:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 16:55:21,958 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676307320910 2023-02-13 16:55:21,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676307320910 2023-02-13 16:55:21,959 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676307320910 2023-02-13 16:55:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,961 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,961 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-13 16:55:21,961 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676307320910 2023-02-13 16:55:21,961 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:21,961 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1676307320910 2023-02-13 16:55:21,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1676307320910 because node does not exist (not an error) 2023-02-13 16:55:21,962 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-13 16:55:21,961 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676307320910 2023-02-13 16:55:21,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-13 16:55:21,961 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-13 16:55:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 16:55:21,962 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676307320910/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:21,962 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1676307320910 2023-02-13 16:55:21,962 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,962 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-13 16:55:21,962 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676307320910 2023-02-13 16:55:21,962 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676307320910 2023-02-13 16:55:21,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-13 16:55:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d] mob files 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 2023-02-13 16:55:21,963 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d 2023-02-13 16:55:21,964 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021359add7626ed24e708979796f06ec3797, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302135b8c91ec7211473189c37abfa5d48e13, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213d4d35bcf268a4258b2601beba113a23e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213e3fc9a6bdedc411f87fbe08abbb0d581, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021335c7444662c4437fb47151ede48018ff, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302136046d46688ba416ba3aa4485e555c6e3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302138e9b9b6df18c47ebbe982f05b25a2079, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302139ef25c70d9d74a318fea3ef635ab895b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302131fca19cf1513448a80061b9d20df2148, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302132ade7814c9294d99a942fe57260cab93, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e69eebf3bf1f423785cd144cfb6762e9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213f91b8ea431d749d9921fd54813e91e39] mob files 2023-02-13 16:55:21,964 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021359add7626ed24e708979796f06ec3797 2023-02-13 16:55:21,964 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302135b8c91ec7211473189c37abfa5d48e13 2023-02-13 16:55:21,964 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213d4d35bcf268a4258b2601beba113a23e 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213e3fc9a6bdedc411f87fbe08abbb0d581 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021335c7444662c4437fb47151ede48018ff 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302136046d46688ba416ba3aa4485e555c6e3 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302138e9b9b6df18c47ebbe982f05b25a2079 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302139ef25c70d9d74a318fea3ef635ab895b 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302131fca19cf1513448a80061b9d20df2148 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302132ade7814c9294d99a942fe57260cab93 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e69eebf3bf1f423785cd144cfb6762e9 2023-02-13 16:55:21,965 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213f91b8ea431d749d9921fd54813e91e39 2023-02-13 16:55:22,057 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:22,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:22,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:22,058 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-13 16:55:22,369 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1676307320910 2023-02-13 16:55:22,371 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/snaptb-1676307320910 2023-02-13 16:55:22,812 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/snaptb-1676307320910 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/snaptb-1676307320910 2023-02-13 16:55:23,058 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:23,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:23,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:23,059 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-13 16:55:23,626 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1676307320910 of table testCompactionWithHFileLink completed 2023-02-13 16:55:23,626 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/snaptb-1676307320910 2023-02-13 16:55:23,627 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/snaptb-1676307320910 2023-02-13 16:55:23,627 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1676307320910 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676307320946Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1676307322369 (+1423 ms)Consolidate snapshot: snaptb-1676307320910 at 1676307322369Loading Region manifests for snaptb-1676307320910 at 1676307322370 (+1 ms)Writing data manifest for snaptb-1676307320910 at 1676307322379 (+9 ms)Verifying snapshot: snaptb-1676307320910 at 1676307322785 (+406 ms)Snapshot snaptb-1676307320910 of table testCompactionWithHFileLink completed at 1676307323626 (+841 ms) 2023-02-13 16:55:23,628 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 16:55:23,629 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.6810 sec 2023-02-13 16:55:24,072 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:55:25,059 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:25,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:25,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-13 16:55:25,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:25,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:25,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 16:55:25,091 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 16:55:25,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:25,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 16:55:25,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 16:55:25,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:25,092 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:25,093 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:25,093 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:25,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:25,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:25,094 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:25,094 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:25,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:25,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 16:55:25,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 16:55:25,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 16:55:25,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:25,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 16:55:25,095 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:25,095 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. started... 2023-02-13 16:55:25,095 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:25,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:25,095 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:25,095 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. started... 2023-02-13 16:55:25,095 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:25,095 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a5218e16a3f58aacc78158f34c1f6d13 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:55:25,095 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 039a7376bb8abccd044c10092dd55a94 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:55:25,095 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. started... 2023-02-13 16:55:25,095 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing cf586ed2ad20940a6fec159be48a9844 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:55:25,911 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/c0abb2acc47c4ffaa986da6997571754 2023-02-13 16:55:25,911 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/2468186692f64cebb201182fd2b08075 2023-02-13 16:55:25,911 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/68e4da02d60a4b5b8ae336a38205505b 2023-02-13 16:55:25,915 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0abb2acc47c4ffaa986da6997571754 2023-02-13 16:55:25,915 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2468186692f64cebb201182fd2b08075 2023-02-13 16:55:25,916 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68e4da02d60a4b5b8ae336a38205505b 2023-02-13 16:55:26,742 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/a327eda999a541828cbc2930f8a30bf6 2023-02-13 16:55:26,743 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/f5f945aeb27f4fec8d347a452201588f 2023-02-13 16:55:26,744 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/d24e5db4a14a4ea6aa928af6b2baf997 2023-02-13 16:55:26,745 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a327eda999a541828cbc2930f8a30bf6 2023-02-13 16:55:26,746 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/c0abb2acc47c4ffaa986da6997571754 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c0abb2acc47c4ffaa986da6997571754 2023-02-13 16:55:26,746 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5f945aeb27f4fec8d347a452201588f 2023-02-13 16:55:26,747 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/2468186692f64cebb201182fd2b08075 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/2468186692f64cebb201182fd2b08075 2023-02-13 16:55:26,747 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d24e5db4a14a4ea6aa928af6b2baf997 2023-02-13 16:55:26,748 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/68e4da02d60a4b5b8ae336a38205505b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/68e4da02d60a4b5b8ae336a38205505b 2023-02-13 16:55:26,750 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0abb2acc47c4ffaa986da6997571754 2023-02-13 16:55:26,750 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c0abb2acc47c4ffaa986da6997571754, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 16:55:26,751 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2468186692f64cebb201182fd2b08075 2023-02-13 16:55:26,751 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/2468186692f64cebb201182fd2b08075, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 16:55:26,751 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/a327eda999a541828cbc2930f8a30bf6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a327eda999a541828cbc2930f8a30bf6 2023-02-13 16:55:26,752 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68e4da02d60a4b5b8ae336a38205505b 2023-02-13 16:55:26,752 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/68e4da02d60a4b5b8ae336a38205505b, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 16:55:26,752 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/f5f945aeb27f4fec8d347a452201588f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f5f945aeb27f4fec8d347a452201588f 2023-02-13 16:55:26,752 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/d24e5db4a14a4ea6aa928af6b2baf997 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/d24e5db4a14a4ea6aa928af6b2baf997 2023-02-13 16:55:26,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a327eda999a541828cbc2930f8a30bf6 2023-02-13 16:55:26,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a327eda999a541828cbc2930f8a30bf6, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 16:55:26,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5f945aeb27f4fec8d347a452201588f 2023-02-13 16:55:26,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f5f945aeb27f4fec8d347a452201588f, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 16:55:26,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a5218e16a3f58aacc78158f34c1f6d13 in 1661ms, sequenceid=24, compaction requested=true 2023-02-13 16:55:26,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for cf586ed2ad20940a6fec159be48a9844 in 1661ms, sequenceid=24, compaction requested=true 2023-02-13 16:55:26,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:26,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:26,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:26,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:26,758 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d24e5db4a14a4ea6aa928af6b2baf997 2023-02-13 16:55:26,758 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/d24e5db4a14a4ea6aa928af6b2baf997, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 16:55:26,759 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 039a7376bb8abccd044c10092dd55a94 in 1664ms, sequenceid=24, compaction requested=true 2023-02-13 16:55:26,759 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:26,759 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 16:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 16:55:26,761 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,761 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:26,761 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,761 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 16:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35047423[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:26,762 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 16:55:26,762 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,766 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 16:55:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 16:55:26,767 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:26,767 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,767 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:26,769 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 16:55:26,769 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,769 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:26,769 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 16:55:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 16:55:26,771 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 16:55:26,771 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:26,771 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:26,771 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 16:55:26,771 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:26,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,773 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:26,773 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:26,774 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 16:55:26,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:26,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:26,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:26,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 16:55:26,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:26,776 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 16:55:26,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:26,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:26,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:26,776 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 16:55:26,777 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 16:55:26,876 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:26,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:26,878 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:26,878 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): 039a7376bb8abccd044c10092dd55a94/family1 is initiating major compaction (all files) 2023-02-13 16:55:26,878 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:26,878 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:26,879 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of 039a7376bb8abccd044c10092dd55a94/family1 in testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:26,879 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/68e4da02d60a4b5b8ae336a38205505b] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp, totalSize=26.7 K 2023-02-13 16:55:26,879 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9eb60b70ae754b51a22135f04efc35d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314140 2023-02-13 16:55:26,880 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting adc5d5b07a8c4a119fa1aa17c983d6b9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:26,880 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting b8332ec2185f4bf29c4897a09af546fb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:26,880 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 78ee9a9d7b274418a710aaa7b139efbe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:26,881 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 68e4da02d60a4b5b8ae336a38205505b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 16:55:26,893 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): 039a7376bb8abccd044c10092dd55a94#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:27,300 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del 2023-02-13 16:55:27,300 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del 2023-02-13 16:55:28,113 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 209a51f9927b4c5cb675147b20309ec1 2023-02-13 16:55:28,114 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/209a51f9927b4c5cb675147b20309ec1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/209a51f9927b4c5cb675147b20309ec1 2023-02-13 16:55:28,117 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 209a51f9927b4c5cb675147b20309ec1 2023-02-13 16:55:28,118 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 039a7376bb8abccd044c10092dd55a94/family1 of 039a7376bb8abccd044c10092dd55a94 into 209a51f9927b4c5cb675147b20309ec1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:28,118 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:28,118 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:28,119 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): 039a7376bb8abccd044c10092dd55a94/family2 is initiating major compaction (all files) 2023-02-13 16:55:28,119 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:28,119 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:28,119 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of 039a7376bb8abccd044c10092dd55a94/family2 in testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:28,119 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/d24e5db4a14a4ea6aa928af6b2baf997] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp, totalSize=25.4 K 2023-02-13 16:55:28,119 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 168873072c92404289dc5f8f0e25e5d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314140 2023-02-13 16:55:28,120 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 87e141fd145f4df4a134a3ed29c37476, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:28,120 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting c3a279f58e664601bf85861d9822473b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:28,120 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9b80ab619d004318a0c72ad5f64a2dd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:28,121 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting d24e5db4a14a4ea6aa928af6b2baf997, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 16:55:28,131 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): 039a7376bb8abccd044c10092dd55a94#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:28,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/68e4da02d60a4b5b8ae336a38205505b] to archive 2023-02-13 16:55:28,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:28,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/9eb60b70ae754b51a22135f04efc35d6 2023-02-13 16:55:28,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/adc5d5b07a8c4a119fa1aa17c983d6b9 2023-02-13 16:55:28,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/b8332ec2185f4bf29c4897a09af546fb 2023-02-13 16:55:28,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/78ee9a9d7b274418a710aaa7b139efbe 2023-02-13 16:55:28,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/68e4da02d60a4b5b8ae336a38205505b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/68e4da02d60a4b5b8ae336a38205505b 2023-02-13 16:55:28,540 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del 2023-02-13 16:55:28,540 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del 2023-02-13 16:55:29,352 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 82cdd99ceb334da3a3604b381a0bf50c 2023-02-13 16:55:29,353 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/82cdd99ceb334da3a3604b381a0bf50c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/82cdd99ceb334da3a3604b381a0bf50c 2023-02-13 16:55:29,356 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 82cdd99ceb334da3a3604b381a0bf50c 2023-02-13 16:55:29,357 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 039a7376bb8abccd044c10092dd55a94/family2 of 039a7376bb8abccd044c10092dd55a94 into 82cdd99ceb334da3a3604b381a0bf50c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:29,357 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:29,357 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:29,357 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): cf586ed2ad20940a6fec159be48a9844/family1 is initiating major compaction (all files) 2023-02-13 16:55:29,358 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:29,358 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:29,358 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of cf586ed2ad20940a6fec159be48a9844/family1 in testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:29,358 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/2468186692f64cebb201182fd2b08075] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp, totalSize=26.7 K 2023-02-13 16:55:29,359 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 018e35dc694942c0bfd1e84103d963a8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314141 2023-02-13 16:55:29,359 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 299fc7a5672047c697fc48979fca6ca1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:29,359 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9d97f86755984e3f8da69d2eb9150580, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:29,360 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 79c464dcbeae4ad8b0567e964b3957c6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:29,360 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 2468186692f64cebb201182fd2b08075, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 16:55:29,370 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): cf586ed2ad20940a6fec159be48a9844#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:29,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/d24e5db4a14a4ea6aa928af6b2baf997] to archive 2023-02-13 16:55:29,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:29,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/168873072c92404289dc5f8f0e25e5d3 2023-02-13 16:55:29,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/87e141fd145f4df4a134a3ed29c37476 2023-02-13 16:55:29,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/c3a279f58e664601bf85861d9822473b 2023-02-13 16:55:29,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/9b80ab619d004318a0c72ad5f64a2dd7 2023-02-13 16:55:29,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/d24e5db4a14a4ea6aa928af6b2baf997 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/d24e5db4a14a4ea6aa928af6b2baf997 2023-02-13 16:55:29,787 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del 2023-02-13 16:55:29,787 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del 2023-02-13 16:55:30,600 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c76a5bb52e9e49028cca768f31f29f4d 2023-02-13 16:55:30,601 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/c76a5bb52e9e49028cca768f31f29f4d as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/c76a5bb52e9e49028cca768f31f29f4d 2023-02-13 16:55:30,604 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c76a5bb52e9e49028cca768f31f29f4d 2023-02-13 16:55:30,605 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cf586ed2ad20940a6fec159be48a9844/family1 of cf586ed2ad20940a6fec159be48a9844 into c76a5bb52e9e49028cca768f31f29f4d(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:30,605 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:30,605 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:30,605 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): cf586ed2ad20940a6fec159be48a9844/family2 is initiating major compaction (all files) 2023-02-13 16:55:30,605 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:30,605 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:30,606 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of cf586ed2ad20940a6fec159be48a9844/family2 in testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:30,606 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f5f945aeb27f4fec8d347a452201588f] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp, totalSize=25.4 K 2023-02-13 16:55:30,606 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 0bbeac208b0c42539fa029aff69fc201, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314141 2023-02-13 16:55:30,606 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting f6ea0d3ea6194f0690b02fe9f4eee202, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:30,606 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 542e06fb7daf47e48c2be60727e2cf0c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:30,607 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting dc077abab99a4109b4db3728d7e0b2ae, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:30,607 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting f5f945aeb27f4fec8d347a452201588f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 16:55:30,617 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): cf586ed2ad20940a6fec159be48a9844#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:30,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/2468186692f64cebb201182fd2b08075] to archive 2023-02-13 16:55:30,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:30,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/018e35dc694942c0bfd1e84103d963a8 2023-02-13 16:55:30,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/299fc7a5672047c697fc48979fca6ca1 2023-02-13 16:55:30,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/9d97f86755984e3f8da69d2eb9150580 2023-02-13 16:55:30,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/79c464dcbeae4ad8b0567e964b3957c6 2023-02-13 16:55:30,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/2468186692f64cebb201182fd2b08075 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/2468186692f64cebb201182fd2b08075 2023-02-13 16:55:31,024 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del 2023-02-13 16:55:31,025 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del 2023-02-13 16:55:31,838 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d78c624fcc2e463289ceb43a3070093f 2023-02-13 16:55:31,839 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/d78c624fcc2e463289ceb43a3070093f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/d78c624fcc2e463289ceb43a3070093f 2023-02-13 16:55:31,842 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d78c624fcc2e463289ceb43a3070093f 2023-02-13 16:55:31,843 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cf586ed2ad20940a6fec159be48a9844/family2 of cf586ed2ad20940a6fec159be48a9844 into d78c624fcc2e463289ceb43a3070093f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:31,843 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:31,844 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:31,844 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): a5218e16a3f58aacc78158f34c1f6d13/family1 is initiating major compaction (all files) 2023-02-13 16:55:31,844 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:31,844 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:31,845 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of a5218e16a3f58aacc78158f34c1f6d13/family1 in testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:31,845 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c0abb2acc47c4ffaa986da6997571754] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp, totalSize=26.7 K 2023-02-13 16:55:31,845 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 4659535d245c4680bb0a591c028508ad, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314138 2023-02-13 16:55:31,846 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9a900594a63745a1a09e0a6288d70be3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:31,846 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 8b5754ecd7b64c539ea5bf7491cbf4b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317324 2023-02-13 16:55:31,846 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 438d44af37cf495b92d1e9d2a48f2d6a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319131 2023-02-13 16:55:31,847 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting c0abb2acc47c4ffaa986da6997571754, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 16:55:31,857 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): a5218e16a3f58aacc78158f34c1f6d13#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:31,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f5f945aeb27f4fec8d347a452201588f] to archive 2023-02-13 16:55:31,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:31,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/0bbeac208b0c42539fa029aff69fc201 2023-02-13 16:55:31,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f6ea0d3ea6194f0690b02fe9f4eee202 2023-02-13 16:55:31,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/542e06fb7daf47e48c2be60727e2cf0c 2023-02-13 16:55:31,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/dc077abab99a4109b4db3728d7e0b2ae 2023-02-13 16:55:31,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f5f945aeb27f4fec8d347a452201588f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/f5f945aeb27f4fec8d347a452201588f 2023-02-13 16:55:32,264 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del 2023-02-13 16:55:32,264 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del 2023-02-13 16:55:33,077 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfd67a901ef94285bd1bfc7cb7b7059f 2023-02-13 16:55:33,078 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/bfd67a901ef94285bd1bfc7cb7b7059f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/bfd67a901ef94285bd1bfc7cb7b7059f 2023-02-13 16:55:33,082 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfd67a901ef94285bd1bfc7cb7b7059f 2023-02-13 16:55:33,083 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a5218e16a3f58aacc78158f34c1f6d13/family1 of a5218e16a3f58aacc78158f34c1f6d13 into bfd67a901ef94285bd1bfc7cb7b7059f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:33,083 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:33,083 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:33,083 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): a5218e16a3f58aacc78158f34c1f6d13/family2 is initiating major compaction (all files) 2023-02-13 16:55:33,083 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:33,083 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:33,083 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of a5218e16a3f58aacc78158f34c1f6d13/family2 in testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:33,083 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a327eda999a541828cbc2930f8a30bf6] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp, totalSize=25.4 K 2023-02-13 16:55:33,084 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 0a88ddb76fe5482797752b8c11c57fb0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314138 2023-02-13 16:55:33,084 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 22a5bb35b55349b29dfcda07f6d868d9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:33,084 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 6a1c1bf3d4c34889a706fff2bac1051c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317324 2023-02-13 16:55:33,085 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting c235dcb64d0940f2be7603e3a595f6f7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319131 2023-02-13 16:55:33,085 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting a327eda999a541828cbc2930f8a30bf6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 16:55:33,097 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): a5218e16a3f58aacc78158f34c1f6d13#family2#compaction#112 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:33,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c0abb2acc47c4ffaa986da6997571754] to archive 2023-02-13 16:55:33,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:33,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/4659535d245c4680bb0a591c028508ad 2023-02-13 16:55:33,145 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/9a900594a63745a1a09e0a6288d70be3 2023-02-13 16:55:33,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/8b5754ecd7b64c539ea5bf7491cbf4b0 2023-02-13 16:55:33,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/438d44af37cf495b92d1e9d2a48f2d6a 2023-02-13 16:55:33,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c0abb2acc47c4ffaa986da6997571754 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c0abb2acc47c4ffaa986da6997571754 2023-02-13 16:55:33,505 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del 2023-02-13 16:55:33,506 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del 2023-02-13 16:55:34,319 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a389cd685bc5463999260b3195959458 2023-02-13 16:55:34,320 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/a389cd685bc5463999260b3195959458 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a389cd685bc5463999260b3195959458 2023-02-13 16:55:34,324 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a389cd685bc5463999260b3195959458 2023-02-13 16:55:34,324 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a5218e16a3f58aacc78158f34c1f6d13/family2 of a5218e16a3f58aacc78158f34c1f6d13 into a389cd685bc5463999260b3195959458(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:34,325 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:34,329 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:34,333 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:34,337 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:34,338 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:34,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a327eda999a541828cbc2930f8a30bf6] to archive 2023-02-13 16:55:34,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/0a88ddb76fe5482797752b8c11c57fb0 2023-02-13 16:55:34,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/22a5bb35b55349b29dfcda07f6d868d9 2023-02-13 16:55:34,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/6a1c1bf3d4c34889a706fff2bac1051c 2023-02-13 16:55:34,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/c235dcb64d0940f2be7603e3a595f6f7 2023-02-13 16:55:34,346 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 16:55:34,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a327eda999a541828cbc2930f8a30bf6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a327eda999a541828cbc2930f8a30bf6 2023-02-13 16:55:34,377 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-13 16:55:34,380 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del 2023-02-13 16:55:34,383 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del 2023-02-13 16:55:34,386 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del 2023-02-13 16:55:34,386 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,387 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(139): Connect 0x3ed89200 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:34,392 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e0a523f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:34,392 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,393 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,396 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,855 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021330c77e3b4ec84eb98cb4df26fb6186d0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021330c77e3b4ec84eb98cb4df26fb6186d0 2023-02-13 16:55:34,855 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021366bd070de0594364837b81ea8403e7ef to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021366bd070de0594364837b81ea8403e7ef 2023-02-13 16:55:34,856 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021314fc130a98f2470da2853431c1bb1cf3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021314fc130a98f2470da2853431c1bb1cf3 2023-02-13 16:55:34,856 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,856 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,857 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,870 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:34,873 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59334, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:34,879 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:55:34,881 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33838, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:55:34,894 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b888f17247344058b1c1927a35c9576c first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-13 16:55:34,894 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/678a8fb1f4764f018bfc40346a666291 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-13 16:55:34,894 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/ee885dd677c84405ae855353866b67b8 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-13 16:55:34,897 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b888f17247344058b1c1927a35c9576c}] 2023-02-13 16:55:34,897 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/ee885dd677c84405ae855353866b67b8}] 2023-02-13 16:55:34,897 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/678a8fb1f4764f018bfc40346a666291}] 2023-02-13 16:55:34,904 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/ee885dd677c84405ae855353866b67b8 for inclusion in a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:34,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/678a8fb1f4764f018bfc40346a666291 for inclusion in cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:34,905 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b888f17247344058b1c1927a35c9576c for inclusion in 039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 16:55:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/678a8fb1f4764f018bfc40346a666291 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tl4f7rnf7l7oum25dsr36a3kd6u3qe0qfa329ger202gcujvvipse8r5m975lu6e/family1/678a8fb1f4764f018bfc40346a666291 2023-02-13 16:55:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/ee885dd677c84405ae855353866b67b8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__oion0dqvbhe7cdgugas02h0bb5rbsvs5f9o8k5jrospt3cqpf1dtcaqr1cqmuekn/family1/ee885dd677c84405ae855353866b67b8 2023-02-13 16:55:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b888f17247344058b1c1927a35c9576c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tf2lqq0eark0ls2lgvutljtsjuu1q8eppkhvbn9not0tt7rhat0hkdmm7is7r3hc/family1/b888f17247344058b1c1927a35c9576c 2023-02-13 16:55:34,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tl4f7rnf7l7oum25dsr36a3kd6u3qe0qfa329ger202gcujvvipse8r5m975lu6e/family1/678a8fb1f4764f018bfc40346a666291 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_ 2023-02-13 16:55:34,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tf2lqq0eark0ls2lgvutljtsjuu1q8eppkhvbn9not0tt7rhat0hkdmm7is7r3hc/family1/b888f17247344058b1c1927a35c9576c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_ 2023-02-13 16:55:34,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__oion0dqvbhe7cdgugas02h0bb5rbsvs5f9o8k5jrospt3cqpf1dtcaqr1cqmuekn/family1/ee885dd677c84405ae855353866b67b8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_ 2023-02-13 16:55:34,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tl4f7rnf7l7oum25dsr36a3kd6u3qe0qfa329ger202gcujvvipse8r5m975lu6e/family1/678a8fb1f4764f018bfc40346a666291 into cf586ed2ad20940a6fec159be48a9844/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_ - updating store file list. 2023-02-13 16:55:34,911 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tf2lqq0eark0ls2lgvutljtsjuu1q8eppkhvbn9not0tt7rhat0hkdmm7is7r3hc/family1/b888f17247344058b1c1927a35c9576c into 039a7376bb8abccd044c10092dd55a94/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_ - updating store file list. 2023-02-13 16:55:34,911 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__oion0dqvbhe7cdgugas02h0bb5rbsvs5f9o8k5jrospt3cqpf1dtcaqr1cqmuekn/family1/ee885dd677c84405ae855353866b67b8 into a5218e16a3f58aacc78158f34c1f6d13/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_ - updating store file list. 2023-02-13 16:55:34,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_ into 039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:34,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tf2lqq0eark0ls2lgvutljtsjuu1q8eppkhvbn9not0tt7rhat0hkdmm7is7r3hc/family1/b888f17247344058b1c1927a35c9576c into 039a7376bb8abccd044c10092dd55a94/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_) 2023-02-13 16:55:34,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_ into cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:34,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tl4f7rnf7l7oum25dsr36a3kd6u3qe0qfa329ger202gcujvvipse8r5m975lu6e/family1/678a8fb1f4764f018bfc40346a666291 into cf586ed2ad20940a6fec159be48a9844/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_) 2023-02-13 16:55:34,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tf2lqq0eark0ls2lgvutljtsjuu1q8eppkhvbn9not0tt7rhat0hkdmm7is7r3hc/family1/b888f17247344058b1c1927a35c9576c 2023-02-13 16:55:34,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__tl4f7rnf7l7oum25dsr36a3kd6u3qe0qfa329ger202gcujvvipse8r5m975lu6e/family1/678a8fb1f4764f018bfc40346a666291 2023-02-13 16:55:34,917 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,917 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,918 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,918 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,920 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df 2023-02-13 16:55:34,920 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e 2023-02-13 16:55:34,920 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_ into a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:34,920 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__oion0dqvbhe7cdgugas02h0bb5rbsvs5f9o8k5jrospt3cqpf1dtcaqr1cqmuekn/family1/ee885dd677c84405ae855353866b67b8 into a5218e16a3f58aacc78158f34c1f6d13/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_) 2023-02-13 16:55:34,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__oion0dqvbhe7cdgugas02h0bb5rbsvs5f9o8k5jrospt3cqpf1dtcaqr1cqmuekn/family1/ee885dd677c84405ae855353866b67b8 2023-02-13 16:55:34,921 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe 2023-02-13 16:55:34,921 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 2023-02-13 16:55:34,922 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a 2023-02-13 16:55:34,922 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e 2023-02-13 16:55:34,923 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 2023-02-13 16:55:34,923 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 16:55:34,923 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 16:55:34,923 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe 2023-02-13 16:55:34,923 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 16:55:34,923 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 16:55:34,924 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,924 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,933 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,934 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,935 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 2023-02-13 16:55:34,936 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 2023-02-13 16:55:34,937 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 2023-02-13 16:55:34,938 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d 2023-02-13 16:55:34,938 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 16:55:34,938 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 16:55:34,938 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,938 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:34,938 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:34,939 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del] 2023-02-13 16:55:34,939 DEBUG [Listener at localhost/36833] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,940 DEBUG [Listener at localhost/36833] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e24d743ffef74ce2a1f01e31b53d9003_del 2023-02-13 16:55:34,940 INFO [Listener at localhost/36833] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 16:55:34,940 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del] 2023-02-13 16:55:34,941 DEBUG [Listener at localhost/36833] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,942 DEBUG [Listener at localhost/36833] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213215bcf9c815a47f1b4b2bbaecef3344f_del 2023-02-13 16:55:34,942 INFO [Listener at localhost/36833] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 16:55:34,942 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del] 2023-02-13 16:55:34,942 DEBUG [Listener at localhost/36833] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:34,944 DEBUG [Listener at localhost/36833] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021360512bce296a4659a2458bc1db5b7fe8_del 2023-02-13 16:55:34,944 INFO [Listener at localhost/36833] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 16:55:34,946 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:34,949 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:34,953 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:34,953 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:34,959 INFO [Listener at localhost/36833] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-13 16:55:34,959 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-13 16:55:34,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-13 16:55:34,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:34,962 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307334962"}]},"ts":"1676307334962"} 2023-02-13 16:55:34,963 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-13 16:55:34,966 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-13 16:55:34,967 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, UNASSIGN}] 2023-02-13 16:55:34,968 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, UNASSIGN 2023-02-13 16:55:34,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, UNASSIGN 2023-02-13 16:55:34,968 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, UNASSIGN 2023-02-13 16:55:34,969 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=a5218e16a3f58aacc78158f34c1f6d13, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:34,969 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=cf586ed2ad20940a6fec159be48a9844, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:34,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=039a7376bb8abccd044c10092dd55a94, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:34,969 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307334969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307334969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307334969"}]},"ts":"1676307334969"} 2023-02-13 16:55:34,969 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307334969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307334969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307334969"}]},"ts":"1676307334969"} 2023-02-13 16:55:34,969 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307334969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307334969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307334969"}]},"ts":"1676307334969"} 2023-02-13 16:55:34,970 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure a5218e16a3f58aacc78158f34c1f6d13, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:34,971 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=91, state=RUNNABLE; CloseRegionProcedure cf586ed2ad20940a6fec159be48a9844, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:34,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=90, state=RUNNABLE; CloseRegionProcedure 039a7376bb8abccd044c10092dd55a94, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:35,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:35,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:35,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cf586ed2ad20940a6fec159be48a9844, disabling compactions & flushes 2023-02-13 16:55:35,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:35,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:35,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. after waiting 0 ms 2023-02-13 16:55:35,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:35,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 16:55:35,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:35,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cf586ed2ad20940a6fec159be48a9844: Waiting for close lock at 1676307335122Running coprocessor pre-close hooks at 1676307335122Disabling compacts and flushes for region at 1676307335122Disabling writes for close at 1676307335122Writing region close event to WAL at 1676307335124 (+2 ms)Running coprocessor post-close hooks at 1676307335127 (+3 ms)Closed at 1676307335128 (+1 ms) 2023-02-13 16:55:35,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:35,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:35,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a5218e16a3f58aacc78158f34c1f6d13, disabling compactions & flushes 2023-02-13 16:55:35,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:35,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:35,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. after waiting 0 ms 2023-02-13 16:55:35,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:35,130 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=cf586ed2ad20940a6fec159be48a9844, regionState=CLOSED 2023-02-13 16:55:35,130 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307335129"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307335129"}]},"ts":"1676307335129"} 2023-02-13 16:55:35,133 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=91 2023-02-13 16:55:35,133 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=91, state=SUCCESS; CloseRegionProcedure cf586ed2ad20940a6fec159be48a9844, server=jenkins-hbase4.apache.org,33907,1676307219708 in 160 msec 2023-02-13 16:55:35,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 16:55:35,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:35,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a5218e16a3f58aacc78158f34c1f6d13: Waiting for close lock at 1676307335129Running coprocessor pre-close hooks at 1676307335129Disabling compacts and flushes for region at 1676307335129Disabling writes for close at 1676307335129Writing region close event to WAL at 1676307335130 (+1 ms)Running coprocessor post-close hooks at 1676307335134 (+4 ms)Closed at 1676307335134 2023-02-13 16:55:35,134 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, UNASSIGN in 166 msec 2023-02-13 16:55:35,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:35,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:35,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 039a7376bb8abccd044c10092dd55a94, disabling compactions & flushes 2023-02-13 16:55:35,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:35,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:35,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. after waiting 0 ms 2023-02-13 16:55:35,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:35,136 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=a5218e16a3f58aacc78158f34c1f6d13, regionState=CLOSED 2023-02-13 16:55:35,136 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307335136"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307335136"}]},"ts":"1676307335136"} 2023-02-13 16:55:35,138 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-02-13 16:55:35,138 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure a5218e16a3f58aacc78158f34c1f6d13, server=jenkins-hbase4.apache.org,33907,1676307219708 in 167 msec 2023-02-13 16:55:35,139 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, UNASSIGN in 171 msec 2023-02-13 16:55:35,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 16:55:35,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:35,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 039a7376bb8abccd044c10092dd55a94: Waiting for close lock at 1676307335136Running coprocessor pre-close hooks at 1676307335136Disabling compacts and flushes for region at 1676307335136Disabling writes for close at 1676307335136Writing region close event to WAL at 1676307335137 (+1 ms)Running coprocessor post-close hooks at 1676307335140 (+3 ms)Closed at 1676307335140 2023-02-13 16:55:35,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:35,142 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=039a7376bb8abccd044c10092dd55a94, regionState=CLOSED 2023-02-13 16:55:35,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307335142"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307335142"}]},"ts":"1676307335142"} 2023-02-13 16:55:35,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=90 2023-02-13 16:55:35,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=90, state=SUCCESS; CloseRegionProcedure 039a7376bb8abccd044c10092dd55a94, server=jenkins-hbase4.apache.org,33907,1676307219708 in 172 msec 2023-02-13 16:55:35,145 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-02-13 16:55:35,145 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, UNASSIGN in 177 msec 2023-02-13 16:55:35,145 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307335145"}]},"ts":"1676307335145"} 2023-02-13 16:55:35,146 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-13 16:55:35,149 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-13 16:55:35,150 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 190 msec 2023-02-13 16:55:35,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 16:55:35,264 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-13 16:55:35,274 INFO [Listener at localhost/36833] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:35,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:35,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676307335275 (current time:1676307335275). 2023-02-13 16:55:35,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-13 16:55:35,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] zookeeper.ReadOnlyZKClient(139): Connect 0x4371061e to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:35,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20bb98e4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:35,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:35,283 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59338, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:35,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4371061e to 127.0.0.1:64619 2023-02-13 16:55:35,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:35,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-13 16:55:35,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:35,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 16:55:35,287 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 16:55:35,287 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676307935287 2023-02-13 16:55:35,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:35,287 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1676307320910-1676307335274 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-13 16:55:35,288 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:35,288 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-13 16:55:35,388 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:35,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:35,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:35,389 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-13 16:55:35,589 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:35,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:35,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:35,590 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-13 16:55:35,693 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 16:55:35,695 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/209a51f9927b4c5cb675147b20309ec1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_] hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/209a51f9927b4c5cb675147b20309ec1 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_ 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/c76a5bb52e9e49028cca768f31f29f4d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_] hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/c76a5bb52e9e49028cca768f31f29f4d 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_ 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/bfd67a901ef94285bd1bfc7cb7b7059f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_] hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/bfd67a901ef94285bd1bfc7cb7b7059f 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_ 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021314fc130a98f2470da2853431c1bb1cf3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021330c77e3b4ec84eb98cb4df26fb6186d0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021366bd070de0594364837b81ea8403e7ef] hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021314fc130a98f2470da2853431c1bb1cf3 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021330c77e3b4ec84eb98cb4df26fb6186d0 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021366bd070de0594364837b81ea8403e7ef 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/82cdd99ceb334da3a3604b381a0bf50c] hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/d78c624fcc2e463289ceb43a3070093f] hfiles 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/82cdd99ceb334da3a3604b381a0bf50c 2023-02-13 16:55:35,698 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/d78c624fcc2e463289ceb43a3070093f 2023-02-13 16:55:35,699 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a389cd685bc5463999260b3195959458] hfiles 2023-02-13 16:55:35,699 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a389cd685bc5463999260b3195959458 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021359add7626ed24e708979796f06ec3797, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302135b8c91ec7211473189c37abfa5d48e13, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213d4d35bcf268a4258b2601beba113a23e, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213e3fc9a6bdedc411f87fbe08abbb0d581, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021335c7444662c4437fb47151ede48018ff, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302136046d46688ba416ba3aa4485e555c6e3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302138e9b9b6df18c47ebbe982f05b25a2079, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302139ef25c70d9d74a318fea3ef635ab895b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302131fca19cf1513448a80061b9d20df2148, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302132ade7814c9294d99a942fe57260cab93, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e69eebf3bf1f423785cd144cfb6762e9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213f91b8ea431d749d9921fd54813e91e39] hfiles 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021359add7626ed24e708979796f06ec3797 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302135b8c91ec7211473189c37abfa5d48e13 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213d4d35bcf268a4258b2601beba113a23e 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213e3fc9a6bdedc411f87fbe08abbb0d581 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021335c7444662c4437fb47151ede48018ff 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302136046d46688ba416ba3aa4485e555c6e3 2023-02-13 16:55:35,704 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302138e9b9b6df18c47ebbe982f05b25a2079 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302139ef25c70d9d74a318fea3ef635ab895b 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302131fca19cf1513448a80061b9d20df2148 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302132ade7814c9294d99a942fe57260cab93 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e69eebf3bf1f423785cd144cfb6762e9 2023-02-13 16:55:35,705 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213f91b8ea431d749d9921fd54813e91e39 2023-02-13 16:55:35,890 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:35,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:35,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:35,892 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-13 16:55:36,117 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-13 16:55:36,117 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:36,118 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:36,392 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:36,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:36,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 16:55:36,393 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-13 16:55:36,547 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676307320910-1676307335274 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:36,561 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1676307320910-1676307335274 of table testCompactionWithHFileLink completed 2023-02-13 16:55:36,561 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:36,561 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:36,561 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1676307320910-1676307335274 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676307335287Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1676307335693 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1676307320910-1676307335274 at 1676307336117 (+424 ms)Loading Region manifests for hbase-failsafe-snaptb-1676307320910-1676307335274 at 1676307336118 (+1 ms)Writing data manifest for hbase-failsafe-snaptb-1676307320910-1676307335274 at 1676307336121 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1676307320910-1676307335274 at 1676307336531 (+410 ms)Snapshot hbase-failsafe-snaptb-1676307320910-1676307335274 of table testCompactionWithHFileLink completed at 1676307336561 (+30 ms) 2023-02-13 16:55:36,563 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 16:55:36,564 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.2770 sec 2023-02-13 16:55:37,393 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 16:55:37,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 16:55:37,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1676307320910-1676307335274 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-13 16:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676307320910" table: "testCompactionWithHFileLink" creation_time: 1676307320920 type: FLUSH version: 2 ttl: 0 ) 2023-02-13 16:55:37,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1676307320910 as table=testCompactionWithHFileLink 2023-02-13 16:55:37,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:37,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:37,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:37,841 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-13 16:55:37,842 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:37,842 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-13 16:55:37,843 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 16:55:37,847 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1676307320910" table: "testCompactionWithHFileLink" creation_time: 1676307320920 type: FLUSH version: 2 ttl: 0 2023-02-13 16:55:37,847 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink 2023-02-13 16:55:37,852 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-13 16:55:37,854 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021314fc130a98f2470da2853431c1bb1cf3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021314fc130a98f2470da2853431c1bb1cf3 2023-02-13 16:55:37,855 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021366bd070de0594364837b81ea8403e7ef to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021366bd070de0594364837b81ea8403e7ef 2023-02-13 16:55:37,856 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021330c77e3b4ec84eb98cb4df26fb6186d0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021330c77e3b4ec84eb98cb4df26fb6186d0 2023-02-13 16:55:37,856 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,859 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,861 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,863 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,866 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,868 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,871 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,873 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,876 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,879 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,881 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,884 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d of snapshot=snaptb-1676307320910 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 16:55:37,889 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213844f97652bae44bfa2dea3c5f97736ee_del 2023-02-13 16:55:37,890 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d89161c84fcc488198596c43f85c6194_del 2023-02-13 16:55:37,891 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302137516791f01914b4aae0776f4e031ab89_del 2023-02-13 16:55:37,891 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:37,891 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:37,891 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:37,894 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/fc60bdf0cbb848eabb5dbce0ec17fe07_SeqId_29_ 2023-02-13 16:55:37,895 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/209a51f9927b4c5cb675147b20309ec1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/209a51f9927b4c5cb675147b20309ec1 2023-02-13 16:55:37,895 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/bfd67a901ef94285bd1bfc7cb7b7059f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/bfd67a901ef94285bd1bfc7cb7b7059f 2023-02-13 16:55:37,896 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/c76a5bb52e9e49028cca768f31f29f4d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/c76a5bb52e9e49028cca768f31f29f4d 2023-02-13 16:55:37,896 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 018e35dc694942c0bfd1e84103d963a8 of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,896 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/366f00c8e05e493fa21635887c0c9268_SeqId_29_ 2023-02-13 16:55:37,897 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9eb60b70ae754b51a22135f04efc35d6 of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,897 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/c27c48a848794477925eb37698db7913_SeqId_29_ 2023-02-13 16:55:37,897 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4659535d245c4680bb0a591c028508ad of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,903 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a900594a63745a1a09e0a6288d70be3 of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,903 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink adc5d5b07a8c4a119fa1aa17c983d6b9 of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,904 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 299fc7a5672047c697fc48979fca6ca1 of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,907 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8b5754ecd7b64c539ea5bf7491cbf4b0 of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,907 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b8332ec2185f4bf29c4897a09af546fb of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,907 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9d97f86755984e3f8da69d2eb9150580 of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,910 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 438d44af37cf495b92d1e9d2a48f2d6a of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,910 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 79c464dcbeae4ad8b0567e964b3957c6 of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,911 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 78ee9a9d7b274418a710aaa7b139efbe of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,916 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/d78c624fcc2e463289ceb43a3070093f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/d78c624fcc2e463289ceb43a3070093f 2023-02-13 16:55:37,916 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0bbeac208b0c42539fa029aff69fc201 of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,916 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a389cd685bc5463999260b3195959458 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/a389cd685bc5463999260b3195959458 2023-02-13 16:55:37,916 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0a88ddb76fe5482797752b8c11c57fb0 of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,916 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/82cdd99ceb334da3a3604b381a0bf50c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/82cdd99ceb334da3a3604b381a0bf50c 2023-02-13 16:55:37,916 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 168873072c92404289dc5f8f0e25e5d3 of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,919 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f6ea0d3ea6194f0690b02fe9f4eee202 of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,919 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 22a5bb35b55349b29dfcda07f6d868d9 of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,919 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 87e141fd145f4df4a134a3ed29c37476 of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,922 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 542e06fb7daf47e48c2be60727e2cf0c of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,923 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6a1c1bf3d4c34889a706fff2bac1051c of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,923 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c3a279f58e664601bf85861d9822473b of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,926 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c235dcb64d0940f2be7603e3a595f6f7 of snapshot=snaptb-1676307320910 to region=a5218e16a3f58aacc78158f34c1f6d13 table=testCompactionWithHFileLink 2023-02-13 16:55:37,926 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dc077abab99a4109b4db3728d7e0b2ae of snapshot=snaptb-1676307320910 to region=cf586ed2ad20940a6fec159be48a9844 table=testCompactionWithHFileLink 2023-02-13 16:55:37,926 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9b80ab619d004318a0c72ad5f64a2dd7 of snapshot=snaptb-1676307320910 to region=039a7376bb8abccd044c10092dd55a94 table=testCompactionWithHFileLink 2023-02-13 16:55:37,929 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1676307320910" table: "testCompactionWithHFileLink" creation_time: 1676307320920 type: FLUSH version: 2 ttl: 0 2023-02-13 16:55:37,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307337930"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:37,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307337930"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:37,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676307337930"}]},"ts":"9223372036854775807"} 2023-02-13 16:55:37,932 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 16:55:37,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 039a7376bb8abccd044c10092dd55a94, NAME => 'testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => a5218e16a3f58aacc78158f34c1f6d13, NAME => 'testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => cf586ed2ad20940a6fec159be48a9844, NAME => 'testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.', STARTKEY => '2', ENDKEY => ''}] 2023-02-13 16:55:37,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307337931"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307337931"}]},"ts":"1676307337931"} 2023-02-13 16:55:37,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307337931"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307337931"}]},"ts":"1676307337931"} 2023-02-13 16:55:37,933 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307337931"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307337931"}]},"ts":"1676307337931"} 2023-02-13 16:55:37,934 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 16:55:37,934 INFO [PEWorker-1] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-13 16:55:37,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 039a7376bb8abccd044c10092dd55a94, NAME => 'testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => a5218e16a3f58aacc78158f34c1f6d13, NAME => 'testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => cf586ed2ad20940a6fec159be48a9844, NAME => 'testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.', STARTKEY => '2', ENDKEY => ''}] 2023-02-13 16:55:37,934 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1676307320910 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-13 16:55:37,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676307320910" table: "testCompactionWithHFileLink" creation_time: 1676307320920 type: FLUSH version: 2 ttl: 0 ) in 512 msec 2023-02-13 16:55:38,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 16:55:38,037 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-13 16:55:38,037 INFO [Listener at localhost/36833] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:38,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1676307320910-1676307335274" 2023-02-13 16:55:38,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1676307320910-1676307335274 2023-02-13 16:55:38,046 INFO [Listener at localhost/36833] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-13 16:55:38,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-13 16:55:38,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-13 16:55:38,058 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-13 16:55:38,058 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307338058"}]},"ts":"1676307338058"} 2023-02-13 16:55:38,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 16:55:38,059 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-13 16:55:38,062 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-13 16:55:38,063 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN}] 2023-02-13 16:55:38,064 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN 2023-02-13 16:55:38,064 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN 2023-02-13 16:55:38,064 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN 2023-02-13 16:55:38,064 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 16:55:38,064 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 16:55:38,064 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 16:55:38,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 16:55:38,216 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=039a7376bb8abccd044c10092dd55a94, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:38,216 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a5218e16a3f58aacc78158f34c1f6d13, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:38,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=cf586ed2ad20940a6fec159be48a9844, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:38,216 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307338216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307338216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307338216"}]},"ts":"1676307338216"} 2023-02-13 16:55:38,216 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307338216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307338216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307338216"}]},"ts":"1676307338216"} 2023-02-13 16:55:38,216 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307338216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307338216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307338216"}]},"ts":"1676307338216"} 2023-02-13 16:55:38,218 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure 039a7376bb8abccd044c10092dd55a94, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:38,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure cf586ed2ad20940a6fec159be48a9844, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:38,219 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure a5218e16a3f58aacc78158f34c1f6d13, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:38,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 16:55:38,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:38,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 039a7376bb8abccd044c10092dd55a94, NAME => 'testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 16:55:38,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:38,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,373 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,374 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:38,374 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:38,374 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 039a7376bb8abccd044c10092dd55a94 columnFamilyName family1 2023-02-13 16:55:38,385 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-78ee9a9d7b274418a710aaa7b139efbe 2023-02-13 16:55:38,390 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9eb60b70ae754b51a22135f04efc35d6 2023-02-13 16:55:38,394 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-adc5d5b07a8c4a119fa1aa17c983d6b9 2023-02-13 16:55:38,399 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-b8332ec2185f4bf29c4897a09af546fb 2023-02-13 16:55:38,399 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(310): Store=039a7376bb8abccd044c10092dd55a94/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:38,399 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,400 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2 2023-02-13 16:55:38,400 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2 2023-02-13 16:55:38,401 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 039a7376bb8abccd044c10092dd55a94 columnFamilyName family2 2023-02-13 16:55:38,406 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-168873072c92404289dc5f8f0e25e5d3 2023-02-13 16:55:38,411 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-87e141fd145f4df4a134a3ed29c37476 2023-02-13 16:55:38,417 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9b80ab619d004318a0c72ad5f64a2dd7 2023-02-13 16:55:38,422 DEBUG [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-c3a279f58e664601bf85861d9822473b 2023-02-13 16:55:38,422 INFO [StoreOpener-039a7376bb8abccd044c10092dd55a94-1] regionserver.HStore(310): Store=039a7376bb8abccd044c10092dd55a94/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:38,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:38,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:55:38,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 039a7376bb8abccd044c10092dd55a94; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73417739, jitterRate=0.09400956332683563}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:38,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 039a7376bb8abccd044c10092dd55a94: Running coprocessor pre-open hook at 1676307338372Writing region info on filesystem at 1676307338372Initializing all the Stores at 1676307338373 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307338373Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307338373Cleaning up temporary data from old regions at 1676307338425 (+52 ms)Cleaning up detritus from prior splits at 1676307338425Running coprocessor post-open hooks at 1676307338428 (+3 ms)Region opened successfully at 1676307338428 2023-02-13 16:55:38,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94., pid=101, masterSystemTime=1676307338369 2023-02-13 16:55:38,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:38,429 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:55:38,429 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:55:38,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:38,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:38,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:38,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:38,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cf586ed2ad20940a6fec159be48a9844, NAME => 'testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 16:55:38,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=039a7376bb8abccd044c10092dd55a94, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:38,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307338431"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307338431"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307338431"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307338431"}]},"ts":"1676307338431"} 2023-02-13 16:55:38,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:38,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,432 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 16:55:38,432 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1912): 039a7376bb8abccd044c10092dd55a94/family1 is initiating minor compaction (all files) 2023-02-13 16:55:38,432 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 039a7376bb8abccd044c10092dd55a94/family1 in testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:38,432 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 16:55:38,432 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9eb60b70ae754b51a22135f04efc35d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-adc5d5b07a8c4a119fa1aa17c983d6b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-b8332ec2185f4bf29c4897a09af546fb, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-78ee9a9d7b274418a710aaa7b139efbe] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp, totalSize=21.6 K 2023-02-13 16:55:38,432 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1912): 039a7376bb8abccd044c10092dd55a94/family2 is initiating minor compaction (all files) 2023-02-13 16:55:38,432 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 039a7376bb8abccd044c10092dd55a94/family2 in testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:55:38,432 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-168873072c92404289dc5f8f0e25e5d3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-87e141fd145f4df4a134a3ed29c37476, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-c3a279f58e664601bf85861d9822473b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9b80ab619d004318a0c72ad5f64a2dd7] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp, totalSize=20.4 K 2023-02-13 16:55:38,432 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,433 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9eb60b70ae754b51a22135f04efc35d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314140 2023-02-13 16:55:38,433 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-168873072c92404289dc5f8f0e25e5d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314140 2023-02-13 16:55:38,433 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-adc5d5b07a8c4a119fa1aa17c983d6b9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:38,433 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:38,434 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:38,434 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-87e141fd145f4df4a134a3ed29c37476, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:38,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-02-13 16:55:38,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure 039a7376bb8abccd044c10092dd55a94, server=jenkins-hbase4.apache.org,33907,1676307219708 in 214 msec 2023-02-13 16:55:38,434 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cf586ed2ad20940a6fec159be48a9844 columnFamilyName family1 2023-02-13 16:55:38,434 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-b8332ec2185f4bf29c4897a09af546fb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:38,435 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-c3a279f58e664601bf85861d9822473b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:38,435 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-78ee9a9d7b274418a710aaa7b139efbe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:38,435 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=039a7376bb8abccd044c10092dd55a94, ASSIGN in 372 msec 2023-02-13 16:55:38,435 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9b80ab619d004318a0c72ad5f64a2dd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:38,447 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-018e35dc694942c0bfd1e84103d963a8 2023-02-13 16:55:38,455 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-299fc7a5672047c697fc48979fca6ca1 2023-02-13 16:55:38,461 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-79c464dcbeae4ad8b0567e964b3957c6 2023-02-13 16:55:38,465 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 039a7376bb8abccd044c10092dd55a94#family1#compaction#113 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:38,473 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] throttle.PressureAwareThroughputController(145): 039a7376bb8abccd044c10092dd55a94#family2#compaction#114 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:38,476 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-9d97f86755984e3f8da69d2eb9150580 2023-02-13 16:55:38,480 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(310): Store=cf586ed2ad20940a6fec159be48a9844/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:38,480 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,481 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2 2023-02-13 16:55:38,481 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2 2023-02-13 16:55:38,481 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cf586ed2ad20940a6fec159be48a9844 columnFamilyName family2 2023-02-13 16:55:38,487 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-0bbeac208b0c42539fa029aff69fc201 2023-02-13 16:55:38,491 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-542e06fb7daf47e48c2be60727e2cf0c 2023-02-13 16:55:38,495 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-dc077abab99a4109b4db3728d7e0b2ae 2023-02-13 16:55:38,499 DEBUG [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-f6ea0d3ea6194f0690b02fe9f4eee202 2023-02-13 16:55:38,499 INFO [StoreOpener-cf586ed2ad20940a6fec159be48a9844-1] regionserver.HStore(310): Store=cf586ed2ad20940a6fec159be48a9844/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:38,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:38,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:55:38,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cf586ed2ad20940a6fec159be48a9844; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59652840, jitterRate=-0.11110341548919678}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:38,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cf586ed2ad20940a6fec159be48a9844: Running coprocessor pre-open hook at 1676307338431Writing region info on filesystem at 1676307338431Initializing all the Stores at 1676307338432 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307338432Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307338432Cleaning up temporary data from old regions at 1676307338501 (+69 ms)Cleaning up detritus from prior splits at 1676307338501Running coprocessor post-open hooks at 1676307338504 (+3 ms)Region opened successfully at 1676307338504 2023-02-13 16:55:38,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844., pid=102, masterSystemTime=1676307338369 2023-02-13 16:55:38,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:55:38,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 16:55:38,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:38,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:38,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:38,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a5218e16a3f58aacc78158f34c1f6d13, NAME => 'testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 16:55:38,506 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=cf586ed2ad20940a6fec159be48a9844, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:38,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:38,506 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307338506"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307338506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307338506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307338506"}]},"ts":"1676307338506"} 2023-02-13 16:55:38,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,508 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,509 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:38,509 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:38,509 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-02-13 16:55:38,509 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure cf586ed2ad20940a6fec159be48a9844, server=jenkins-hbase4.apache.org,33907,1676307219708 in 290 msec 2023-02-13 16:55:38,509 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5218e16a3f58aacc78158f34c1f6d13 columnFamilyName family1 2023-02-13 16:55:38,510 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=cf586ed2ad20940a6fec159be48a9844, ASSIGN in 447 msec 2023-02-13 16:55:38,516 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-438d44af37cf495b92d1e9d2a48f2d6a 2023-02-13 16:55:38,521 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-4659535d245c4680bb0a591c028508ad 2023-02-13 16:55:38,527 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-8b5754ecd7b64c539ea5bf7491cbf4b0 2023-02-13 16:55:38,533 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-9a900594a63745a1a09e0a6288d70be3 2023-02-13 16:55:38,533 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(310): Store=a5218e16a3f58aacc78158f34c1f6d13/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:38,534 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,535 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2 2023-02-13 16:55:38,535 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2 2023-02-13 16:55:38,535 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5218e16a3f58aacc78158f34c1f6d13 columnFamilyName family2 2023-02-13 16:55:38,542 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-0a88ddb76fe5482797752b8c11c57fb0 2023-02-13 16:55:38,551 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-22a5bb35b55349b29dfcda07f6d868d9 2023-02-13 16:55:38,558 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-6a1c1bf3d4c34889a706fff2bac1051c 2023-02-13 16:55:38,563 DEBUG [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(539): loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-c235dcb64d0940f2be7603e3a595f6f7 2023-02-13 16:55:38,564 INFO [StoreOpener-a5218e16a3f58aacc78158f34c1f6d13-1] regionserver.HStore(310): Store=a5218e16a3f58aacc78158f34c1f6d13/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:38,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:38,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:55:38,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a5218e16a3f58aacc78158f34c1f6d13; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65565193, jitterRate=-0.023002490401268005}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:38,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a5218e16a3f58aacc78158f34c1f6d13: Running coprocessor pre-open hook at 1676307338507Writing region info on filesystem at 1676307338507Initializing all the Stores at 1676307338507Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307338507Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307338508 (+1 ms)Cleaning up temporary data from old regions at 1676307338566 (+58 ms)Cleaning up detritus from prior splits at 1676307338566Running coprocessor post-open hooks at 1676307338568 (+2 ms)Region opened successfully at 1676307338568 2023-02-13 16:55:38,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13., pid=103, masterSystemTime=1676307338369 2023-02-13 16:55:38,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 16:55:38,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 16:55:38,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:38,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:38,571 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a5218e16a3f58aacc78158f34c1f6d13, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:38,571 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676307338571"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307338571"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307338571"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307338571"}]},"ts":"1676307338571"} 2023-02-13 16:55:38,574 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-02-13 16:55:38,574 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure a5218e16a3f58aacc78158f34c1f6d13, server=jenkins-hbase4.apache.org,33907,1676307219708 in 354 msec 2023-02-13 16:55:38,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=98, resume processing ppid=97 2023-02-13 16:55:38,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a5218e16a3f58aacc78158f34c1f6d13, ASSIGN in 512 msec 2023-02-13 16:55:38,576 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307338576"}]},"ts":"1676307338576"} 2023-02-13 16:55:38,577 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-13 16:55:38,578 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-13 16:55:38,580 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 528 msec 2023-02-13 16:55:38,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 16:55:38,662 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-13 16:55:38,676 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:38,685 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:38,695 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:38,697 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:38,703 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 16:55:38,718 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-13 16:55:38,718 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:38,718 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(139): Connect 0x28b43b23 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:38,723 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28189b88, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:38,723 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:38,723 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:38,723 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:39,291 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family1/19840ecc892b431ba51e77dc60ad8427 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/19840ecc892b431ba51e77dc60ad8427 2023-02-13 16:55:39,295 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 16:55:39,295 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 039a7376bb8abccd044c10092dd55a94/family1 of 039a7376bb8abccd044c10092dd55a94 into 19840ecc892b431ba51e77dc60ad8427(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:55:39,295 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:39,295 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94., storeName=039a7376bb8abccd044c10092dd55a94/family1, priority=12, startTime=1676307338428; duration=0sec 2023-02-13 16:55:39,295 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 16:55:39,296 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:55:39,298 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 16:55:39,298 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1912): cf586ed2ad20940a6fec159be48a9844/family1 is initiating minor compaction (all files) 2023-02-13 16:55:39,298 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cf586ed2ad20940a6fec159be48a9844/family1 in testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:39,298 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-018e35dc694942c0bfd1e84103d963a8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-299fc7a5672047c697fc48979fca6ca1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-9d97f86755984e3f8da69d2eb9150580, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-79c464dcbeae4ad8b0567e964b3957c6] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp, totalSize=21.6 K 2023-02-13 16:55:39,298 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-018e35dc694942c0bfd1e84103d963a8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314141 2023-02-13 16:55:39,299 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-299fc7a5672047c697fc48979fca6ca1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:39,299 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-9d97f86755984e3f8da69d2eb9150580, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:39,300 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-79c464dcbeae4ad8b0567e964b3957c6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:39,311 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): cf586ed2ad20940a6fec159be48a9844#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:39,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9eb60b70ae754b51a22135f04efc35d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-adc5d5b07a8c4a119fa1aa17c983d6b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-b8332ec2185f4bf29c4897a09af546fb, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-78ee9a9d7b274418a710aaa7b139efbe] to archive 2023-02-13 16:55:39,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:39,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9eb60b70ae754b51a22135f04efc35d6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9eb60b70ae754b51a22135f04efc35d6 2023-02-13 16:55:39,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-adc5d5b07a8c4a119fa1aa17c983d6b9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-adc5d5b07a8c4a119fa1aa17c983d6b9 2023-02-13 16:55:39,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-b8332ec2185f4bf29c4897a09af546fb to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-b8332ec2185f4bf29c4897a09af546fb 2023-02-13 16:55:39,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-78ee9a9d7b274418a710aaa7b139efbe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-78ee9a9d7b274418a710aaa7b139efbe 2023-02-13 16:55:39,597 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213388679148b324d0895d05e64f30ce0e0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213388679148b324d0895d05e64f30ce0e0 2023-02-13 16:55:39,597 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021368e21352d56b471db918498a0da3037e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021368e21352d56b471db918498a0da3037e 2023-02-13 16:55:39,597 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139c2ecd919dca4691967adf96196dac43 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139c2ecd919dca4691967adf96196dac43 2023-02-13 16:55:39,598 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,598 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,598 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,611 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:39,615 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59346, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:39,621 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:55:39,633 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33844, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:55:39,650 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/7030816bb5504641b3115505a31fb6db first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-13 16:55:39,650 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/c90b21ac9e4b4aecb07c1397ec7e3291 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-13 16:55:39,651 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=32 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/7030816bb5504641b3115505a31fb6db}] 2023-02-13 16:55:39,651 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/c90b21ac9e4b4aecb07c1397ec7e3291}] 2023-02-13 16:55:39,651 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/59a67582d0c94c7394f91988b6365c50 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-13 16:55:39,652 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/59a67582d0c94c7394f91988b6365c50}] 2023-02-13 16:55:39,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/c90b21ac9e4b4aecb07c1397ec7e3291 for inclusion in 039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:39,659 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/59a67582d0c94c7394f91988b6365c50 for inclusion in cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:39,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/7030816bb5504641b3115505a31fb6db for inclusion in a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:39,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-13 16:55:39,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 16:55:39,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-13 16:55:39,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 16:55:39,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:39,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:39,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-13 16:55:39,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 16:55:39,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:39,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/c90b21ac9e4b4aecb07c1397ec7e3291 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__kb3o8hg57jpl8nppepbi6tg04kckjlt57ec5j5i4vgba51lebjeb6gjkop2gk3ab/family1/c90b21ac9e4b4aecb07c1397ec7e3291 2023-02-13 16:55:39,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/7030816bb5504641b3115505a31fb6db to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__mgtvptt764v05ibcou01ae9h2risat6il462t3dro9ma3c8s7h6qcfoo3797qb24/family1/7030816bb5504641b3115505a31fb6db 2023-02-13 16:55:39,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/59a67582d0c94c7394f91988b6365c50 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__a4du01bjresgtduqm1o4pm3rntl6jfjlguskm1pfrvfmni9fafacq7ji8k22r2pa/family1/59a67582d0c94c7394f91988b6365c50 2023-02-13 16:55:39,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__kb3o8hg57jpl8nppepbi6tg04kckjlt57ec5j5i4vgba51lebjeb6gjkop2gk3ab/family1/c90b21ac9e4b4aecb07c1397ec7e3291 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/42cb511b7d074cf8a1e27123fa022e51_SeqId_35_ 2023-02-13 16:55:39,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__a4du01bjresgtduqm1o4pm3rntl6jfjlguskm1pfrvfmni9fafacq7ji8k22r2pa/family1/59a67582d0c94c7394f91988b6365c50 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/0346198ec90b4a56ac2cac2fcb98b8a1_SeqId_34_ 2023-02-13 16:55:39,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__mgtvptt764v05ibcou01ae9h2risat6il462t3dro9ma3c8s7h6qcfoo3797qb24/family1/7030816bb5504641b3115505a31fb6db as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_ 2023-02-13 16:55:39,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__kb3o8hg57jpl8nppepbi6tg04kckjlt57ec5j5i4vgba51lebjeb6gjkop2gk3ab/family1/c90b21ac9e4b4aecb07c1397ec7e3291 into 039a7376bb8abccd044c10092dd55a94/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/42cb511b7d074cf8a1e27123fa022e51_SeqId_35_ - updating store file list. 2023-02-13 16:55:39,667 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__a4du01bjresgtduqm1o4pm3rntl6jfjlguskm1pfrvfmni9fafacq7ji8k22r2pa/family1/59a67582d0c94c7394f91988b6365c50 into cf586ed2ad20940a6fec159be48a9844/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/0346198ec90b4a56ac2cac2fcb98b8a1_SeqId_34_ - updating store file list. 2023-02-13 16:55:39,668 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__mgtvptt764v05ibcou01ae9h2risat6il462t3dro9ma3c8s7h6qcfoo3797qb24/family1/7030816bb5504641b3115505a31fb6db into a5218e16a3f58aacc78158f34c1f6d13/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_ - updating store file list. 2023-02-13 16:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/42cb511b7d074cf8a1e27123fa022e51_SeqId_35_ into 039a7376bb8abccd044c10092dd55a94/family1 2023-02-13 16:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__kb3o8hg57jpl8nppepbi6tg04kckjlt57ec5j5i4vgba51lebjeb6gjkop2gk3ab/family1/c90b21ac9e4b4aecb07c1397ec7e3291 into 039a7376bb8abccd044c10092dd55a94/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family1/42cb511b7d074cf8a1e27123fa022e51_SeqId_35_) 2023-02-13 16:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_ into a5218e16a3f58aacc78158f34c1f6d13/family1 2023-02-13 16:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__mgtvptt764v05ibcou01ae9h2risat6il462t3dro9ma3c8s7h6qcfoo3797qb24/family1/7030816bb5504641b3115505a31fb6db into a5218e16a3f58aacc78158f34c1f6d13/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_) 2023-02-13 16:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/0346198ec90b4a56ac2cac2fcb98b8a1_SeqId_34_ into cf586ed2ad20940a6fec159be48a9844/family1 2023-02-13 16:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__a4du01bjresgtduqm1o4pm3rntl6jfjlguskm1pfrvfmni9fafacq7ji8k22r2pa/family1/59a67582d0c94c7394f91988b6365c50 into cf586ed2ad20940a6fec159be48a9844/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/0346198ec90b4a56ac2cac2fcb98b8a1_SeqId_34_) 2023-02-13 16:55:39,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__kb3o8hg57jpl8nppepbi6tg04kckjlt57ec5j5i4vgba51lebjeb6gjkop2gk3ab/family1/c90b21ac9e4b4aecb07c1397ec7e3291 2023-02-13 16:55:39,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__mgtvptt764v05ibcou01ae9h2risat6il462t3dro9ma3c8s7h6qcfoo3797qb24/family1/7030816bb5504641b3115505a31fb6db 2023-02-13 16:55:39,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testCompactionWithHFileLink__a4du01bjresgtduqm1o4pm3rntl6jfjlguskm1pfrvfmni9fafacq7ji8k22r2pa/family1/59a67582d0c94c7394f91988b6365c50 2023-02-13 16:55:39,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:39,674 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,675 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:39,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-13 16:55:39,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(1912): a5218e16a3f58aacc78158f34c1f6d13/family1 is initiating minor compaction (all files) 2023-02-13 16:55:39,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3df10973; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 16:55:39,676 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302130481a1d3cd8d442bb704d6a207da92df 2023-02-13 16:55:39,677 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,678 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:39,678 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,678 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302132c82cbb08554436893c49a3c694e4cbe 2023-02-13 16:55:39,679 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:39,680 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302137b29252e54884f5ea05d25f2f396420e 2023-02-13 16:55:39,680 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302139362a0cf5c9443f68139bc04f1e8789a 2023-02-13 16:55:39,681 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302131e6686885b3540bfa2a24acd600791e8 2023-02-13 16:55:39,681 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302139d97ebea5ff64c64b186717b457ae536 2023-02-13 16:55:39,681 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230213c91794243f2349ec9855545216715fb1 2023-02-13 16:55:39,682 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 16:55:39,682 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 16:55:39,682 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:39,682 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213dea54a66359643cfad61558e5e38c00e 2023-02-13 16:55:39,683 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302135f5b1ddbed174696aae6084d90f7f522 2023-02-13 16:55:39,683 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213df4d02f61a5d4e1081a69ff9da690bfe 2023-02-13 16:55:39,684 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 16:55:39,684 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 16:55:39,684 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213916ad271ad484f27b4fe001dd1988b75 2023-02-13 16:55:39,684 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:39,684 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213dc85f6ac44ca40fa80aca02a73b09f9d 2023-02-13 16:55:39,685 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 16:55:39,685 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 16:55:39,685 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 16:55:39,685 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 16:55:39,688 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:39,692 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:39,695 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/.tmp/family2/41645c870d574c2886822edcc6e1e812 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/41645c870d574c2886822edcc6e1e812 2023-02-13 16:55:39,695 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:39,696 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:39,700 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 039a7376bb8abccd044c10092dd55a94/family2 of 039a7376bb8abccd044c10092dd55a94 into 41645c870d574c2886822edcc6e1e812(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:39,700 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 039a7376bb8abccd044c10092dd55a94: 2023-02-13 16:55:39,700 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94., storeName=039a7376bb8abccd044c10092dd55a94/family2, priority=12, startTime=1676307338429; duration=1sec 2023-02-13 16:55:39,700 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 16:55:39,700 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a5218e16a3f58aacc78158f34c1f6d13/family1 in testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:39,700 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-4659535d245c4680bb0a591c028508ad, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-9a900594a63745a1a09e0a6288d70be3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-8b5754ecd7b64c539ea5bf7491cbf4b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-438d44af37cf495b92d1e9d2a48f2d6a, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp, totalSize=28.9 K 2023-02-13 16:55:39,701 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-4659535d245c4680bb0a591c028508ad, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314138 2023-02-13 16:55:39,702 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-9a900594a63745a1a09e0a6288d70be3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:39,702 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-8b5754ecd7b64c539ea5bf7491cbf4b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317324 2023-02-13 16:55:39,703 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-438d44af37cf495b92d1e9d2a48f2d6a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319131 2023-02-13 16:55:39,703 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:39,703 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting cb87cd800cb141b08bf8411acaca5a58_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1676307314138 2023-02-13 16:55:39,704 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:39,722 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] throttle.PressureAwareThroughputController(145): a5218e16a3f58aacc78158f34c1f6d13#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:39,723 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=283 (was 234) - Thread LEAK? -, OpenFileDescriptor=761 (was 597) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=134 (was 158), ProcessCount=168 (was 168), AvailableMemoryMB=1113 (was 1212) 2023-02-13 16:55:39,736 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=286, OpenFileDescriptor=765, MaxFileDescriptor=60000, SystemLoadAverage=134, ProcessCount=168, AvailableMemoryMB=1113 2023-02-13 16:55:39,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-168873072c92404289dc5f8f0e25e5d3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-87e141fd145f4df4a134a3ed29c37476, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-c3a279f58e664601bf85861d9822473b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9b80ab619d004318a0c72ad5f64a2dd7] to archive 2023-02-13 16:55:39,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:39,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-168873072c92404289dc5f8f0e25e5d3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-168873072c92404289dc5f8f0e25e5d3 2023-02-13 16:55:39,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-87e141fd145f4df4a134a3ed29c37476 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-87e141fd145f4df4a134a3ed29c37476 2023-02-13 16:55:39,742 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-13 16:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-13 16:55:39,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-c3a279f58e664601bf85861d9822473b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-c3a279f58e664601bf85861d9822473b 2023-02-13 16:55:39,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9b80ab619d004318a0c72ad5f64a2dd7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/family2/testCompactionWithHFileLink=039a7376bb8abccd044c10092dd55a94-9b80ab619d004318a0c72ad5f64a2dd7 2023-02-13 16:55:39,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-13 16:55:39,748 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 16:55:39,751 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-02-13 16:55:39,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-13 16:55:39,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 16:55:39,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-13 16:55:39,851 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:55:39,851 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-13 16:55:39,852 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:55:39,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:39,853 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:39,853 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:39,853 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255 2023-02-13 16:55:39,854 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88 empty. 2023-02-13 16:55:39,854 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255 empty. 2023-02-13 16:55:39,854 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4 empty. 2023-02-13 16:55:39,854 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:39,854 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255 2023-02-13 16:55:39,854 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:39,854 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-13 16:55:39,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:40,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:40,261 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:40,262 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => d345489d398503e06fb04841004d8255, NAME => 'ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:40,262 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 433223184ab0ee1d5c98c521a6bdfa88, NAME => 'ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:40,262 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => c0c2fdbf028f696dc83568e799a88cc4, NAME => 'ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:40,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:40,527 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family1/f565c99138974872b08cbd48e2b46cd4 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/f565c99138974872b08cbd48e2b46cd4 2023-02-13 16:55:40,531 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in cf586ed2ad20940a6fec159be48a9844/family1 of cf586ed2ad20940a6fec159be48a9844 into f565c99138974872b08cbd48e2b46cd4(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:40,531 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:40,531 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844., storeName=cf586ed2ad20940a6fec159be48a9844/family1, priority=12, startTime=1676307338504; duration=1sec 2023-02-13 16:55:40,531 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 16:55:40,531 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:55:40,532 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 16:55:40,533 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1912): cf586ed2ad20940a6fec159be48a9844/family2 is initiating minor compaction (all files) 2023-02-13 16:55:40,533 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cf586ed2ad20940a6fec159be48a9844/family2 in testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:55:40,533 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-0bbeac208b0c42539fa029aff69fc201, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-f6ea0d3ea6194f0690b02fe9f4eee202, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-542e06fb7daf47e48c2be60727e2cf0c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-dc077abab99a4109b4db3728d7e0b2ae] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp, totalSize=20.4 K 2023-02-13 16:55:40,533 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-0bbeac208b0c42539fa029aff69fc201, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314141 2023-02-13 16:55:40,534 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-f6ea0d3ea6194f0690b02fe9f4eee202, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:40,534 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-542e06fb7daf47e48c2be60727e2cf0c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317325 2023-02-13 16:55:40,535 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-dc077abab99a4109b4db3728d7e0b2ae, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319132 2023-02-13 16:55:40,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-018e35dc694942c0bfd1e84103d963a8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-299fc7a5672047c697fc48979fca6ca1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-9d97f86755984e3f8da69d2eb9150580, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-79c464dcbeae4ad8b0567e964b3957c6] to archive 2023-02-13 16:55:40,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:40,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-018e35dc694942c0bfd1e84103d963a8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-018e35dc694942c0bfd1e84103d963a8 2023-02-13 16:55:40,545 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): cf586ed2ad20940a6fec159be48a9844#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:40,546 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-299fc7a5672047c697fc48979fca6ca1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-299fc7a5672047c697fc48979fca6ca1 2023-02-13 16:55:40,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-9d97f86755984e3f8da69d2eb9150580 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-9d97f86755984e3f8da69d2eb9150580 2023-02-13 16:55:40,548 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-79c464dcbeae4ad8b0567e964b3957c6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family1/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-79c464dcbeae4ad8b0567e964b3957c6 2023-02-13 16:55:40,667 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing c0c2fdbf028f696dc83568e799a88cc4, disabling compactions & flushes 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing d345489d398503e06fb04841004d8255, disabling compactions & flushes 2023-02-13 16:55:40,668 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 433223184ab0ee1d5c98c521a6bdfa88, disabling compactions & flushes 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:40,668 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. after waiting 0 ms 2023-02-13 16:55:40,668 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. after waiting 0 ms 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:40,668 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:40,668 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:40,669 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. after waiting 0 ms 2023-02-13 16:55:40,669 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for c0c2fdbf028f696dc83568e799a88cc4: Waiting for close lock at 1676307340668Disabling compacts and flushes for region at 1676307340668Disabling writes for close at 1676307340668Writing region close event to WAL at 1676307340668Closed at 1676307340668 2023-02-13 16:55:40,668 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:40,669 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:40,669 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 433223184ab0ee1d5c98c521a6bdfa88: Waiting for close lock at 1676307340668Disabling compacts and flushes for region at 1676307340668Disabling writes for close at 1676307340668Writing region close event to WAL at 1676307340668Closed at 1676307340668 2023-02-13 16:55:40,669 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:40,669 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for d345489d398503e06fb04841004d8255: Waiting for close lock at 1676307340668Disabling compacts and flushes for region at 1676307340668Disabling writes for close at 1676307340669 (+1 ms)Writing region close event to WAL at 1676307340669Closed at 1676307340669 2023-02-13 16:55:40,671 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:55:40,672 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676307340672"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307340672"}]},"ts":"1676307340672"} 2023-02-13 16:55:40,672 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676307340672"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307340672"}]},"ts":"1676307340672"} 2023-02-13 16:55:40,672 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676307340672"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307340672"}]},"ts":"1676307340672"} 2023-02-13 16:55:40,674 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 16:55:40,674 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:55:40,675 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307340675"}]},"ts":"1676307340675"} 2023-02-13 16:55:40,676 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-13 16:55:40,678 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=433223184ab0ee1d5c98c521a6bdfa88, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0c2fdbf028f696dc83568e799a88cc4, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d345489d398503e06fb04841004d8255, ASSIGN}] 2023-02-13 16:55:40,680 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0c2fdbf028f696dc83568e799a88cc4, ASSIGN 2023-02-13 16:55:40,681 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d345489d398503e06fb04841004d8255, ASSIGN 2023-02-13 16:55:40,681 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=433223184ab0ee1d5c98c521a6bdfa88, ASSIGN 2023-02-13 16:55:40,681 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0c2fdbf028f696dc83568e799a88cc4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:40,681 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=433223184ab0ee1d5c98c521a6bdfa88, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:40,681 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d345489d398503e06fb04841004d8255, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:40,834 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=d345489d398503e06fb04841004d8255, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:40,834 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=433223184ab0ee1d5c98c521a6bdfa88, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:40,834 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=c0c2fdbf028f696dc83568e799a88cc4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:40,834 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676307340834"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307340834"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307340834"}]},"ts":"1676307340834"} 2023-02-13 16:55:40,834 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676307340834"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307340834"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307340834"}]},"ts":"1676307340834"} 2023-02-13 16:55:40,834 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676307340834"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307340834"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307340834"}]},"ts":"1676307340834"} 2023-02-13 16:55:40,835 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure d345489d398503e06fb04841004d8255, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:40,836 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 433223184ab0ee1d5c98c521a6bdfa88, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:40,836 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure c0c2fdbf028f696dc83568e799a88cc4, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:40,942 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family1/f32b727754ad4967b22890dbbe5f16e1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/f32b727754ad4967b22890dbbe5f16e1 2023-02-13 16:55:40,946 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in a5218e16a3f58aacc78158f34c1f6d13/family1 of a5218e16a3f58aacc78158f34c1f6d13 into f32b727754ad4967b22890dbbe5f16e1(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:40,946 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:40,946 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1676307339676; duration=1sec 2023-02-13 16:55:40,946 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 16:55:40,946 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 16:55:40,949 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 16:55:40,949 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1912): a5218e16a3f58aacc78158f34c1f6d13/family2 is initiating minor compaction (all files) 2023-02-13 16:55:40,949 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a5218e16a3f58aacc78158f34c1f6d13/family2 in testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:55:40,949 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-0a88ddb76fe5482797752b8c11c57fb0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-22a5bb35b55349b29dfcda07f6d868d9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-6a1c1bf3d4c34889a706fff2bac1051c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-c235dcb64d0940f2be7603e3a595f6f7] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp, totalSize=20.4 K 2023-02-13 16:55:40,950 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-0a88ddb76fe5482797752b8c11c57fb0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307314138 2023-02-13 16:55:40,950 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-22a5bb35b55349b29dfcda07f6d868d9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307315529 2023-02-13 16:55:40,951 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-6a1c1bf3d4c34889a706fff2bac1051c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307317324 2023-02-13 16:55:40,951 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-c235dcb64d0940f2be7603e3a595f6f7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307319131 2023-02-13 16:55:40,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:40,963 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] throttle.PressureAwareThroughputController(145): a5218e16a3f58aacc78158f34c1f6d13#family2#compaction#118 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:55:40,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:40,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d345489d398503e06fb04841004d8255, NAME => 'ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 16:55:40,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:40,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,990 INFO [StoreOpener-d345489d398503e06fb04841004d8255-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,991 DEBUG [StoreOpener-d345489d398503e06fb04841004d8255-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1 2023-02-13 16:55:40,991 DEBUG [StoreOpener-d345489d398503e06fb04841004d8255-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1 2023-02-13 16:55:40,992 INFO [StoreOpener-d345489d398503e06fb04841004d8255-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d345489d398503e06fb04841004d8255 columnFamilyName family1 2023-02-13 16:55:40,992 INFO [StoreOpener-d345489d398503e06fb04841004d8255-1] regionserver.HStore(310): Store=d345489d398503e06fb04841004d8255/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:40,993 INFO [StoreOpener-d345489d398503e06fb04841004d8255-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,994 DEBUG [StoreOpener-d345489d398503e06fb04841004d8255-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2 2023-02-13 16:55:40,994 DEBUG [StoreOpener-d345489d398503e06fb04841004d8255-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2 2023-02-13 16:55:40,994 INFO [StoreOpener-d345489d398503e06fb04841004d8255-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d345489d398503e06fb04841004d8255 columnFamilyName family2 2023-02-13 16:55:40,995 INFO [StoreOpener-d345489d398503e06fb04841004d8255-1] regionserver.HStore(310): Store=d345489d398503e06fb04841004d8255/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:40,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:40,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d345489d398503e06fb04841004d8255 2023-02-13 16:55:40,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:40,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d345489d398503e06fb04841004d8255; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69581602, jitterRate=0.036846667528152466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:40,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d345489d398503e06fb04841004d8255: Running coprocessor pre-open hook at 1676307340989Writing region info on filesystem at 1676307340989Initializing all the Stores at 1676307340990 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307340990Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307340990Cleaning up temporary data from old regions at 1676307340996 (+6 ms)Cleaning up detritus from prior splits at 1676307340996Running coprocessor post-open hooks at 1676307340999 (+3 ms)Region opened successfully at 1676307340999 2023-02-13 16:55:41,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255., pid=109, masterSystemTime=1676307340987 2023-02-13 16:55:41,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:41,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:41,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:41,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c0c2fdbf028f696dc83568e799a88cc4, NAME => 'ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 16:55:41,001 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=d345489d398503e06fb04841004d8255, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:41,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,002 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676307341001"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307341001"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307341001"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307341001"}]},"ts":"1676307341001"} 2023-02-13 16:55:41,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:41,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,003 INFO [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,004 DEBUG [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1 2023-02-13 16:55:41,004 DEBUG [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1 2023-02-13 16:55:41,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-02-13 16:55:41,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure d345489d398503e06fb04841004d8255, server=jenkins-hbase4.apache.org,33907,1676307219708 in 168 msec 2023-02-13 16:55:41,004 INFO [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0c2fdbf028f696dc83568e799a88cc4 columnFamilyName family1 2023-02-13 16:55:41,005 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d345489d398503e06fb04841004d8255, ASSIGN in 326 msec 2023-02-13 16:55:41,005 INFO [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] regionserver.HStore(310): Store=c0c2fdbf028f696dc83568e799a88cc4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:41,005 INFO [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,006 DEBUG [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2 2023-02-13 16:55:41,006 DEBUG [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2 2023-02-13 16:55:41,006 INFO [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0c2fdbf028f696dc83568e799a88cc4 columnFamilyName family2 2023-02-13 16:55:41,007 INFO [StoreOpener-c0c2fdbf028f696dc83568e799a88cc4-1] regionserver.HStore(310): Store=c0c2fdbf028f696dc83568e799a88cc4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:41,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:41,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:55:41,010 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.07692307692307693, tune throughput to 53.85 MB/second 2023-02-13 16:55:41,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:41,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c0c2fdbf028f696dc83568e799a88cc4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62862381, jitterRate=-0.06327752768993378}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:41,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c0c2fdbf028f696dc83568e799a88cc4: Running coprocessor pre-open hook at 1676307341002Writing region info on filesystem at 1676307341002Initializing all the Stores at 1676307341002Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307341002Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307341003 (+1 ms)Cleaning up temporary data from old regions at 1676307341008 (+5 ms)Cleaning up detritus from prior splits at 1676307341009 (+1 ms)Running coprocessor post-open hooks at 1676307341013 (+4 ms)Region opened successfully at 1676307341013 2023-02-13 16:55:41,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4., pid=111, masterSystemTime=1676307340987 2023-02-13 16:55:41,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:41,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:41,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:41,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 433223184ab0ee1d5c98c521a6bdfa88, NAME => 'ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 16:55:41,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=c0c2fdbf028f696dc83568e799a88cc4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:41,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:41,015 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676307341015"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307341015"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307341015"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307341015"}]},"ts":"1676307341015"} 2023-02-13 16:55:41,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,016 INFO [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,017 DEBUG [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1 2023-02-13 16:55:41,017 DEBUG [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1 2023-02-13 16:55:41,018 INFO [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 433223184ab0ee1d5c98c521a6bdfa88 columnFamilyName family1 2023-02-13 16:55:41,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-02-13 16:55:41,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure c0c2fdbf028f696dc83568e799a88cc4, server=jenkins-hbase4.apache.org,33907,1676307219708 in 180 msec 2023-02-13 16:55:41,019 INFO [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] regionserver.HStore(310): Store=433223184ab0ee1d5c98c521a6bdfa88/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:41,019 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0c2fdbf028f696dc83568e799a88cc4, ASSIGN in 340 msec 2023-02-13 16:55:41,019 INFO [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,020 DEBUG [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2 2023-02-13 16:55:41,020 DEBUG [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2 2023-02-13 16:55:41,020 INFO [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 433223184ab0ee1d5c98c521a6bdfa88 columnFamilyName family2 2023-02-13 16:55:41,021 INFO [StoreOpener-433223184ab0ee1d5c98c521a6bdfa88-1] regionserver.HStore(310): Store=433223184ab0ee1d5c98c521a6bdfa88/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:41,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:55:41,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:55:41,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:41,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 433223184ab0ee1d5c98c521a6bdfa88; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69831235, jitterRate=0.04056648910045624}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:55:41,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 433223184ab0ee1d5c98c521a6bdfa88: Running coprocessor pre-open hook at 1676307341015Writing region info on filesystem at 1676307341015Initializing all the Stores at 1676307341016 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307341016Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307341016Cleaning up temporary data from old regions at 1676307341022 (+6 ms)Cleaning up detritus from prior splits at 1676307341023 (+1 ms)Running coprocessor post-open hooks at 1676307341026 (+3 ms)Region opened successfully at 1676307341026 2023-02-13 16:55:41,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88., pid=110, masterSystemTime=1676307340987 2023-02-13 16:55:41,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:41,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:41,028 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=433223184ab0ee1d5c98c521a6bdfa88, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:41,028 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676307341028"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307341028"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307341028"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307341028"}]},"ts":"1676307341028"} 2023-02-13 16:55:41,030 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-02-13 16:55:41,030 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 433223184ab0ee1d5c98c521a6bdfa88, server=jenkins-hbase4.apache.org,33907,1676307219708 in 193 msec 2023-02-13 16:55:41,031 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 16:55:41,031 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=433223184ab0ee1d5c98c521a6bdfa88, ASSIGN in 352 msec 2023-02-13 16:55:41,031 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:55:41,031 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307341031"}]},"ts":"1676307341031"} 2023-02-13 16:55:41,032 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-13 16:55:41,034 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:55:41,035 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1850 sec 2023-02-13 16:55:41,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-4659535d245c4680bb0a591c028508ad, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-9a900594a63745a1a09e0a6288d70be3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-8b5754ecd7b64c539ea5bf7491cbf4b0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-438d44af37cf495b92d1e9d2a48f2d6a, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_] to archive 2023-02-13 16:55:41,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:41,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-4659535d245c4680bb0a591c028508ad to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-4659535d245c4680bb0a591c028508ad 2023-02-13 16:55:41,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-9a900594a63745a1a09e0a6288d70be3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-9a900594a63745a1a09e0a6288d70be3 2023-02-13 16:55:41,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-8b5754ecd7b64c539ea5bf7491cbf4b0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-8b5754ecd7b64c539ea5bf7491cbf4b0 2023-02-13 16:55:41,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-438d44af37cf495b92d1e9d2a48f2d6a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-438d44af37cf495b92d1e9d2a48f2d6a 2023-02-13 16:55:41,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family1/cb87cd800cb141b08bf8411acaca5a58_SeqId_34_ 2023-02-13 16:55:41,362 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/.tmp/family2/eaafc5754e4d4cbd997fc697290942b6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/eaafc5754e4d4cbd997fc697290942b6 2023-02-13 16:55:41,366 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in cf586ed2ad20940a6fec159be48a9844/family2 of cf586ed2ad20940a6fec159be48a9844 into eaafc5754e4d4cbd997fc697290942b6(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:55:41,366 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cf586ed2ad20940a6fec159be48a9844: 2023-02-13 16:55:41,366 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844., storeName=cf586ed2ad20940a6fec159be48a9844/family2, priority=12, startTime=1676307338505; duration=0sec 2023-02-13 16:55:41,366 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 16:55:41,366 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 16:55:41,367 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-13 16:55:41,367 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-13 16:55:41,367 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. because compaction request was cancelled 2023-02-13 16:55:41,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-0bbeac208b0c42539fa029aff69fc201, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-f6ea0d3ea6194f0690b02fe9f4eee202, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-542e06fb7daf47e48c2be60727e2cf0c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-dc077abab99a4109b4db3728d7e0b2ae] to archive 2023-02-13 16:55:41,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:41,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-0bbeac208b0c42539fa029aff69fc201 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-0bbeac208b0c42539fa029aff69fc201 2023-02-13 16:55:41,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-f6ea0d3ea6194f0690b02fe9f4eee202 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-f6ea0d3ea6194f0690b02fe9f4eee202 2023-02-13 16:55:41,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-542e06fb7daf47e48c2be60727e2cf0c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-542e06fb7daf47e48c2be60727e2cf0c 2023-02-13 16:55:41,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-dc077abab99a4109b4db3728d7e0b2ae to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/family2/testCompactionWithHFileLink=cf586ed2ad20940a6fec159be48a9844-dc077abab99a4109b4db3728d7e0b2ae 2023-02-13 16:55:41,513 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-13 16:55:41,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 16:55:41,957 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-13 16:55:41,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:41,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:41,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:41,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:41,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 16:55:41,967 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 16:55:41,967 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:41,967 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 16:55:41,967 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 16:55:41,968 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:41,968 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:41,969 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:41,969 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:41,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:41,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:41,969 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:41,969 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:41,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:41,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:41,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:41,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:41,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 16:55:41,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 16:55:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 16:55:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 16:55:41,971 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:41,971 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. started... 2023-02-13 16:55:41,971 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:41,971 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. started... 2023-02-13 16:55:41,971 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:41,973 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 433223184ab0ee1d5c98c521a6bdfa88 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:55:41,973 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:41,973 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:41,973 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0c2fdbf028f696dc83568e799a88cc4 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:55:41,973 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. started... 2023-02-13 16:55:41,973 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d345489d398503e06fb04841004d8255 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:55:42,179 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/.tmp/family2/f5f45e74b59d450ba6529e0852bfb10e as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/f5f45e74b59d450ba6529e0852bfb10e 2023-02-13 16:55:42,183 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in a5218e16a3f58aacc78158f34c1f6d13/family2 of a5218e16a3f58aacc78158f34c1f6d13 into f5f45e74b59d450ba6529e0852bfb10e(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:42,183 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a5218e16a3f58aacc78158f34c1f6d13: 2023-02-13 16:55:42,183 INFO [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13., storeName=a5218e16a3f58aacc78158f34c1f6d13/family2, priority=12, startTime=1676307338569; duration=1sec 2023-02-13 16:55:42,183 DEBUG [RS:0;jenkins-hbase4:33907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:55:42,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-0a88ddb76fe5482797752b8c11c57fb0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-22a5bb35b55349b29dfcda07f6d868d9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-6a1c1bf3d4c34889a706fff2bac1051c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-c235dcb64d0940f2be7603e3a595f6f7] to archive 2023-02-13 16:55:42,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:42,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-0a88ddb76fe5482797752b8c11c57fb0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-0a88ddb76fe5482797752b8c11c57fb0 2023-02-13 16:55:42,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-22a5bb35b55349b29dfcda07f6d868d9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-22a5bb35b55349b29dfcda07f6d868d9 2023-02-13 16:55:42,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-6a1c1bf3d4c34889a706fff2bac1051c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-6a1c1bf3d4c34889a706fff2bac1051c 2023-02-13 16:55:42,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-c235dcb64d0940f2be7603e3a595f6f7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/family2/testCompactionWithHFileLink=a5218e16a3f58aacc78158f34c1f6d13-c235dcb64d0940f2be7603e3a595f6f7 2023-02-13 16:55:42,389 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213b568f899bf244fa29799bc639e0c2a31 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213b568f899bf244fa29799bc639e0c2a31 2023-02-13 16:55:42,389 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021366aba80442274460a4831d316f87ca1b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021366aba80442274460a4831d316f87ca1b 2023-02-13 16:55:42,396 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302138703c6257f3b431bb2d2752910a17db2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302138703c6257f3b431bb2d2752910a17db2 2023-02-13 16:55:42,397 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/e9471fa873754d5882950adf18670105 2023-02-13 16:55:42,798 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/f1b92ef544924afdba942ba8fb28aa02 2023-02-13 16:55:42,801 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/854b5e1cef26478094a6ff7f1b4c69d1 2023-02-13 16:55:42,810 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213a60b8a5cc5114c84a032acb9a5315f6d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213a60b8a5cc5114c84a032acb9a5315f6d 2023-02-13 16:55:43,212 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133677c39b921e4208989bfe4758eec073 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302133677c39b921e4208989bfe4758eec073 2023-02-13 16:55:43,214 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/086860fe730147e2bb5b46e3af260fd6 2023-02-13 16:55:43,214 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213716cf994a64b4facbc9873c92323234c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213716cf994a64b4facbc9873c92323234c 2023-02-13 16:55:43,218 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/e9471fa873754d5882950adf18670105 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/e9471fa873754d5882950adf18670105 2023-02-13 16:55:43,218 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/2ab871df26f144e59e770f8a6e0428e3 2023-02-13 16:55:43,222 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/e9471fa873754d5882950adf18670105, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 16:55:43,222 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/f1b92ef544924afdba942ba8fb28aa02 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f1b92ef544924afdba942ba8fb28aa02 2023-02-13 16:55:43,223 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/086860fe730147e2bb5b46e3af260fd6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/086860fe730147e2bb5b46e3af260fd6 2023-02-13 16:55:43,226 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f1b92ef544924afdba942ba8fb28aa02, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 16:55:43,226 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/086860fe730147e2bb5b46e3af260fd6, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:55:43,227 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/2ab871df26f144e59e770f8a6e0428e3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/2ab871df26f144e59e770f8a6e0428e3 2023-02-13 16:55:43,227 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d345489d398503e06fb04841004d8255 in 1254ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:43,228 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:43,228 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:43,228 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:43,231 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/2ab871df26f144e59e770f8a6e0428e3, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:55:43,231 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 433223184ab0ee1d5c98c521a6bdfa88 in 1258ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:43,231 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:43,231 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:43,231 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:43,620 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/fb87190e155c4e52a4e3cdd74824aefc 2023-02-13 16:55:43,624 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/854b5e1cef26478094a6ff7f1b4c69d1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/854b5e1cef26478094a6ff7f1b4c69d1 2023-02-13 16:55:43,627 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/854b5e1cef26478094a6ff7f1b4c69d1, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 16:55:43,627 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/fb87190e155c4e52a4e3cdd74824aefc as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/fb87190e155c4e52a4e3cdd74824aefc 2023-02-13 16:55:43,630 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/fb87190e155c4e52a4e3cdd74824aefc, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:55:43,631 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c0c2fdbf028f696dc83568e799a88cc4 in 1658ms, sequenceid=5, compaction requested=false 2023-02-13 16:55:43,631 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:43,631 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:43,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:43,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:43,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 16:55:43,631 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 16:55:43,633 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,633 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:43,633 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,633 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:43,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 16:55:43,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5114f00[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:43,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 16:55:43,635 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,636 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 16:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 16:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:43,636 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,636 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:43,639 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,639 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 16:55:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:43,639 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:43,639 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 16:55:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:43,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 16:55:43,641 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 16:55:43,641 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:43,641 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:43,641 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 16:55:43,641 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:43,642 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,642 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,642 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:43,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:43,643 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:43,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:43,644 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:43,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:43,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:43,646 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 16:55:43,646 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:43,646 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:43,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 16:55:43,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:43,646 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:43,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:43,746 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:43,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:43,749 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:43,749 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:43,749 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:43,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:43,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:43,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 16:55:43,753 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 16:55:43,753 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:43,753 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 16:55:43,753 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 16:55:43,753 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,753 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:43,755 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:43,755 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:43,755 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:43,755 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:43,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:43,756 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 16:55:43,756 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 16:55:43,756 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 16:55:43,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 16:55:43,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:43,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 16:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. started... 2023-02-13 16:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:43,757 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:43,757 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0c2fdbf028f696dc83568e799a88cc4 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. started... 2023-02-13 16:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:43,757 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:43,757 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 433223184ab0ee1d5c98c521a6bdfa88 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. started... 2023-02-13 16:55:43,757 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d345489d398503e06fb04841004d8255 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:55:44,172 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213b3237a28f71745faabcddb0083ef155f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213b3237a28f71745faabcddb0083ef155f 2023-02-13 16:55:44,172 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139544f6a8f0474f7d88055b6538c8d7c8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302139544f6a8f0474f7d88055b6538c8d7c8 2023-02-13 16:55:44,172 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213ed28931861064ecf84aea36861a56690 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213ed28931861064ecf84aea36861a56690 2023-02-13 16:55:44,577 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/c6989189d86f48aeb8f4a0d2128333aa 2023-02-13 16:55:44,577 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/a36f3a4e4f984461b30f85f972615fb9 2023-02-13 16:55:44,578 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/93d8322480b745beb13ca54b670f86d1 2023-02-13 16:55:44,599 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213007d32547fe74225b868314096a119ab to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230213007d32547fe74225b868314096a119ab 2023-02-13 16:55:44,997 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213caaa9ca8f88541998ff23840fd16a3c7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213caaa9ca8f88541998ff23840fd16a3c7 2023-02-13 16:55:44,997 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302139ff50b030d114e60bb8a8948fe2f3afd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302139ff50b030d114e60bb8a8948fe2f3afd 2023-02-13 16:55:45,005 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/f94258edd9b9401e9175fd5dcbcf9dc7 2023-02-13 16:55:45,009 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/a36f3a4e4f984461b30f85f972615fb9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/a36f3a4e4f984461b30f85f972615fb9 2023-02-13 16:55:45,012 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/a36f3a4e4f984461b30f85f972615fb9, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 16:55:45,013 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/f94258edd9b9401e9175fd5dcbcf9dc7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/f94258edd9b9401e9175fd5dcbcf9dc7 2023-02-13 16:55:45,016 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/f94258edd9b9401e9175fd5dcbcf9dc7, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:55:45,017 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 433223184ab0ee1d5c98c521a6bdfa88 in 1260ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:45,017 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:45,017 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:45,017 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:45,402 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/8addf97f1b894062b3de6f897c9f68b4 2023-02-13 16:55:45,405 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/16b29f83ffce4cce92c579fcc6db5b8f 2023-02-13 16:55:45,407 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/c6989189d86f48aeb8f4a0d2128333aa as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c6989189d86f48aeb8f4a0d2128333aa 2023-02-13 16:55:45,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/93d8322480b745beb13ca54b670f86d1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/93d8322480b745beb13ca54b670f86d1 2023-02-13 16:55:45,411 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c6989189d86f48aeb8f4a0d2128333aa, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 16:55:45,412 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/8addf97f1b894062b3de6f897c9f68b4 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/8addf97f1b894062b3de6f897c9f68b4 2023-02-13 16:55:45,415 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/8addf97f1b894062b3de6f897c9f68b4, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:55:45,416 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c0c2fdbf028f696dc83568e799a88cc4 in 1659ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:45,416 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:45,416 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:45,416 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:45,417 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/93d8322480b745beb13ca54b670f86d1, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 16:55:45,418 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/16b29f83ffce4cce92c579fcc6db5b8f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/16b29f83ffce4cce92c579fcc6db5b8f 2023-02-13 16:55:45,421 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/16b29f83ffce4cce92c579fcc6db5b8f, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:55:45,421 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d345489d398503e06fb04841004d8255 in 1664ms, sequenceid=9, compaction requested=false 2023-02-13 16:55:45,422 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:45,422 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:45,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:45,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:45,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 16:55:45,422 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 16:55:45,424 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,424 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:45,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:45,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 16:55:45,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2859143b[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:45,426 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 16:55:45,426 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,428 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,428 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 16:55:45,428 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 16:55:45,428 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:45,428 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,428 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:45,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 16:55:45,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:45,432 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,432 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 16:55:45,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:45,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:45,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:45,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:45,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:45,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:45,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:45,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:45,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 16:55:45,434 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 16:55:45,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:45,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:45,434 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 16:55:45,434 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:45,437 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,437 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:45,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:45,437 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:45,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:45,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:45,438 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:45,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:45,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,439 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:45,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:45,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,441 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,441 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:45,441 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:45,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:45,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 16:55:45,441 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:45,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:45,441 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 16:55:45,441 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:45,441 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:45,441 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:45,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:45,442 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:45,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:45,442 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,442 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:45,442 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,442 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:45,442 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,542 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:45,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:45,544 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:45,544 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:45,545 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:45,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:45,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:45,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 16:55:45,549 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 16:55:45,549 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:45,549 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 16:55:45,549 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 16:55:45,549 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,549 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:45,551 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,551 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:45,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:45,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:45,551 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:45,552 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:45,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:45,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:45,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 16:55:45,553 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 16:55:45,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 16:55:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 16:55:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 16:55:45,553 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:45,553 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:45,553 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. started... 2023-02-13 16:55:45,553 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:45,553 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. started... 2023-02-13 16:55:45,553 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. started... 2023-02-13 16:55:45,553 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:45,553 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 433223184ab0ee1d5c98c521a6bdfa88 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:55:45,553 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0c2fdbf028f696dc83568e799a88cc4 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:55:45,553 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d345489d398503e06fb04841004d8255 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:55:45,576 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213531950d3d82d4a2fbd37c424dd6038df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213531950d3d82d4a2fbd37c424dd6038df 2023-02-13 16:55:45,580 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/430ce9d5cd414df49612aff0db1032fa 2023-02-13 16:55:45,978 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213ddf1adafdaf04fbaacba46f5b7c6cbcf to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213ddf1adafdaf04fbaacba46f5b7c6cbcf 2023-02-13 16:55:45,978 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213790c58f846ef49878229ab6c86e7ab48 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213790c58f846ef49878229ab6c86e7ab48 2023-02-13 16:55:45,985 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/9d0dc7420d534f48afdcf7c64ce965d7 2023-02-13 16:55:45,992 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302131001625806384401afbed4b70e0c0d70 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302131001625806384401afbed4b70e0c0d70 2023-02-13 16:55:46,002 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/b5518e29ae90443bb7a5d1b84788a0d6 2023-02-13 16:55:46,005 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/430ce9d5cd414df49612aff0db1032fa as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/430ce9d5cd414df49612aff0db1032fa 2023-02-13 16:55:46,008 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/430ce9d5cd414df49612aff0db1032fa, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 16:55:46,009 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/b5518e29ae90443bb7a5d1b84788a0d6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/b5518e29ae90443bb7a5d1b84788a0d6 2023-02-13 16:55:46,012 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/b5518e29ae90443bb7a5d1b84788a0d6, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:55:46,013 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 433223184ab0ee1d5c98c521a6bdfa88 in 460ms, sequenceid=13, compaction requested=true 2023-02-13 16:55:46,013 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:46,013 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:46,013 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:46,382 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac 2023-02-13 16:55:46,404 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213e9f5166b7bdd4731b0bf4977b5f1dffd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213e9f5166b7bdd4731b0bf4977b5f1dffd 2023-02-13 16:55:46,491 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:55:46,530 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:46,530 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28b43b23 to 127.0.0.1:64619 2023-02-13 16:55:46,530 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:46,530 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:55:46,531 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ed89200 to 127.0.0.1:64619 2023-02-13 16:55:46,531 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:55:46,795 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f8cb7de0624b4776bcb940b9ea20e760 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213f8cb7de0624b4776bcb940b9ea20e760 2023-02-13 16:55:46,808 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/f02fb546b823405fa27d3aa8238612a5 2023-02-13 16:55:46,811 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/9d0dc7420d534f48afdcf7c64ce965d7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/9d0dc7420d534f48afdcf7c64ce965d7 2023-02-13 16:55:46,814 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/9d0dc7420d534f48afdcf7c64ce965d7, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 16:55:46,815 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/f02fb546b823405fa27d3aa8238612a5 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/f02fb546b823405fa27d3aa8238612a5 2023-02-13 16:55:46,817 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/f02fb546b823405fa27d3aa8238612a5, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:55:46,818 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d345489d398503e06fb04841004d8255 in 1265ms, sequenceid=13, compaction requested=true 2023-02-13 16:55:46,818 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:46,818 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:46,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:47,200 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/9e07b9a8d70b489aabe00ca8eb2e82cd 2023-02-13 16:55:47,204 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac 2023-02-13 16:55:47,207 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 16:55:47,207 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/9e07b9a8d70b489aabe00ca8eb2e82cd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/9e07b9a8d70b489aabe00ca8eb2e82cd 2023-02-13 16:55:47,210 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/9e07b9a8d70b489aabe00ca8eb2e82cd, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:55:47,211 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c0c2fdbf028f696dc83568e799a88cc4 in 1658ms, sequenceid=13, compaction requested=true 2023-02-13 16:55:47,211 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:47,211 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:47,211 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:47,211 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:47,211 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,211 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 16:55:47,211 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 16:55:47,213 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,213 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:47,213 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,213 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:47,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:47,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:47,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:47,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:47,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 16:55:47,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68482d58[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:47,215 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 16:55:47,215 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,216 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,216 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 16:55:47,216 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 16:55:47,216 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,217 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:47,216 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:47,218 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,218 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 16:55:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:47,218 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:47,218 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 16:55:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:47,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:47,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:47,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 16:55:47,220 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 16:55:47,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:47,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:47,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 16:55:47,220 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:47,223 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,223 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:47,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,223 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:47,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:47,225 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:47,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:47,227 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 16:55:47,227 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:47,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:47,227 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:47,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:47,227 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:47,228 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,328 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:47,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:47,330 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:47,330 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:47,330 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:55:47,332 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:47,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:47,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 16:55:47,334 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 16:55:47,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:47,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 16:55:47,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 16:55:47,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,334 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:47,335 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,336 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:47,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:47,336 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:47,336 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:47,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:47,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 16:55:47,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 16:55:47,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 16:55:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 16:55:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 16:55:47,337 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:47,337 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. started... 2023-02-13 16:55:47,337 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:47,337 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. started... 2023-02-13 16:55:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:47,337 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0c2fdbf028f696dc83568e799a88cc4 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:55:47,337 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 433223184ab0ee1d5c98c521a6bdfa88 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:55:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:47,337 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:47,337 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. started... 2023-02-13 16:55:47,338 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d345489d398503e06fb04841004d8255 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:55:47,356 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213f224e842819549f787c1be14f7708ba7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213f224e842819549f787c1be14f7708ba7 2023-02-13 16:55:47,749 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021361d77740850743c48cc41d3d0674f5cb to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021361d77740850743c48cc41d3d0674f5cb 2023-02-13 16:55:47,752 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302139f0b36ef2b4a4cd994ab12384e42b8cf to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302139f0b36ef2b4a4cd994ab12384e42b8cf 2023-02-13 16:55:47,761 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/56051bd0c56441df88506d3a20580da8 2023-02-13 16:55:48,153 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/9687ad74fa9f46a2973fa4f4eaaca7ef 2023-02-13 16:55:48,156 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/8eb4eaca2d3741f8a6a805f48be944e6 2023-02-13 16:55:48,165 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302130abb8229d0a548efb5ad5bdcc2c85879 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302130abb8229d0a548efb5ad5bdcc2c85879 2023-02-13 16:55:48,167 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213cfab87fbcd1946f29f09defe97c4b2fd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230213cfab87fbcd1946f29f09defe97c4b2fd 2023-02-13 16:55:48,169 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/220770ea40ed4868ab853aa5abab3fdc 2023-02-13 16:55:48,172 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302137155d7102c764d40a86bbbf3877c2a78 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302137155d7102c764d40a86bbbf3877c2a78 2023-02-13 16:55:48,174 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/9687ad74fa9f46a2973fa4f4eaaca7ef as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/9687ad74fa9f46a2973fa4f4eaaca7ef 2023-02-13 16:55:48,176 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/2e6f9dc9421c4be183b4afa24421b027 2023-02-13 16:55:48,177 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/9687ad74fa9f46a2973fa4f4eaaca7ef, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 16:55:48,177 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/220770ea40ed4868ab853aa5abab3fdc as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/220770ea40ed4868ab853aa5abab3fdc 2023-02-13 16:55:48,179 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/56051bd0c56441df88506d3a20580da8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/56051bd0c56441df88506d3a20580da8 2023-02-13 16:55:48,180 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/220770ea40ed4868ab853aa5abab3fdc, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:55:48,181 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c0c2fdbf028f696dc83568e799a88cc4 in 844ms, sequenceid=17, compaction requested=true 2023-02-13 16:55:48,181 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:48,181 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:48,182 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/56051bd0c56441df88506d3a20580da8, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 16:55:48,183 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/2e6f9dc9421c4be183b4afa24421b027 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/2e6f9dc9421c4be183b4afa24421b027 2023-02-13 16:55:48,186 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/2e6f9dc9421c4be183b4afa24421b027, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:55:48,186 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d345489d398503e06fb04841004d8255 in 848ms, sequenceid=17, compaction requested=true 2023-02-13 16:55:48,187 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:48,187 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:48,187 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:48,572 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/649fb3f2ef51455189e7b98a3549ea0b 2023-02-13 16:55:48,576 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/8eb4eaca2d3741f8a6a805f48be944e6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/8eb4eaca2d3741f8a6a805f48be944e6 2023-02-13 16:55:48,579 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/8eb4eaca2d3741f8a6a805f48be944e6, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 16:55:48,580 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/649fb3f2ef51455189e7b98a3549ea0b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/649fb3f2ef51455189e7b98a3549ea0b 2023-02-13 16:55:48,583 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/649fb3f2ef51455189e7b98a3549ea0b, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:55:48,584 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 433223184ab0ee1d5c98c521a6bdfa88 in 1247ms, sequenceid=17, compaction requested=true 2023-02-13 16:55:48,584 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:48,584 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:48,584 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:48,584 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:48,584 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,584 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 16:55:48,584 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 16:55:48,585 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,585 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:48,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:48,586 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,586 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:48,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:48,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:48,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:48,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:48,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 16:55:48,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8d004a6[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:48,587 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 16:55:48,587 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,590 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 16:55:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 16:55:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:48,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,590 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:48,591 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 16:55:48,591 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,591 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:48,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,591 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 16:55:48,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:48,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:48,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:48,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:48,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 16:55:48,594 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 16:55:48,594 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:48,594 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:48,594 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 16:55:48,594 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:48,595 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,595 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,595 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:48,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:48,596 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:48,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:48,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:48,597 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:48,599 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:48,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 16:55:48,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:48,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:48,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 16:55:48,600 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:48,599 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:48,600 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:48,700 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:48,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:48,710 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:48,718 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:48,727 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:48,727 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:48,750 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:55:48,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 16:55:48,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 16:55:48,752 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 16:55:48,752 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:48,752 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 16:55:48,752 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 16:55:48,752 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,752 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:48,755 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,755 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:48,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:48,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:48,755 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:48,755 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:55:48,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:48,755 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:55:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:48,756 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 16:55:48,756 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 16:55:48,756 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 16:55:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 16:55:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:55:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 16:55:48,756 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:48,756 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. started... 2023-02-13 16:55:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:55:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:55:48,756 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:48,756 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0c2fdbf028f696dc83568e799a88cc4 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:55:48,756 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:48,756 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. started... 2023-02-13 16:55:48,756 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. started... 2023-02-13 16:55:48,757 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 433223184ab0ee1d5c98c521a6bdfa88 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:55:48,757 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d345489d398503e06fb04841004d8255 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:55:49,570 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/c1e8d9e908d148ffbb24e2d2b2d290fb 2023-02-13 16:55:49,570 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/f4bfa4cc3e8840acab1e9a7689d72057 2023-02-13 16:55:49,570 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/ea750dc704514772863469874a3147da 2023-02-13 16:55:49,573 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e8d9e908d148ffbb24e2d2b2d290fb 2023-02-13 16:55:49,574 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea750dc704514772863469874a3147da 2023-02-13 16:55:49,574 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4bfa4cc3e8840acab1e9a7689d72057 2023-02-13 16:55:49,997 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/958fae319e0a4b029e9576523ab7e8ce 2023-02-13 16:55:50,000 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 958fae319e0a4b029e9576523ab7e8ce 2023-02-13 16:55:50,001 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/f4bfa4cc3e8840acab1e9a7689d72057 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f4bfa4cc3e8840acab1e9a7689d72057 2023-02-13 16:55:50,004 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4bfa4cc3e8840acab1e9a7689d72057 2023-02-13 16:55:50,004 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f4bfa4cc3e8840acab1e9a7689d72057, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 16:55:50,004 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/958fae319e0a4b029e9576523ab7e8ce as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/958fae319e0a4b029e9576523ab7e8ce 2023-02-13 16:55:50,007 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 958fae319e0a4b029e9576523ab7e8ce 2023-02-13 16:55:50,007 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/958fae319e0a4b029e9576523ab7e8ce, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 16:55:50,008 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 433223184ab0ee1d5c98c521a6bdfa88 in 1251ms, sequenceid=23, compaction requested=true 2023-02-13 16:55:50,008 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:50,008 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:50,008 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:55:50,396 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/a9a44642c5d240a79bbf778c61c3a7ad 2023-02-13 16:55:50,396 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/b3ff07e63d33489aa1d9072c7a541450 2023-02-13 16:55:50,399 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9a44642c5d240a79bbf778c61c3a7ad 2023-02-13 16:55:50,400 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b3ff07e63d33489aa1d9072c7a541450 2023-02-13 16:55:50,400 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/c1e8d9e908d148ffbb24e2d2b2d290fb as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c1e8d9e908d148ffbb24e2d2b2d290fb 2023-02-13 16:55:50,400 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/ea750dc704514772863469874a3147da as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/ea750dc704514772863469874a3147da 2023-02-13 16:55:50,403 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e8d9e908d148ffbb24e2d2b2d290fb 2023-02-13 16:55:50,403 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea750dc704514772863469874a3147da 2023-02-13 16:55:50,403 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c1e8d9e908d148ffbb24e2d2b2d290fb, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 16:55:50,403 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/ea750dc704514772863469874a3147da, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 16:55:50,404 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/a9a44642c5d240a79bbf778c61c3a7ad as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/a9a44642c5d240a79bbf778c61c3a7ad 2023-02-13 16:55:50,404 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/b3ff07e63d33489aa1d9072c7a541450 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/b3ff07e63d33489aa1d9072c7a541450 2023-02-13 16:55:50,408 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9a44642c5d240a79bbf778c61c3a7ad 2023-02-13 16:55:50,408 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/a9a44642c5d240a79bbf778c61c3a7ad, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 16:55:50,408 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b3ff07e63d33489aa1d9072c7a541450 2023-02-13 16:55:50,408 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/b3ff07e63d33489aa1d9072c7a541450, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 16:55:50,409 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c0c2fdbf028f696dc83568e799a88cc4 in 1653ms, sequenceid=23, compaction requested=true 2023-02-13 16:55:50,409 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d345489d398503e06fb04841004d8255 in 1652ms, sequenceid=23, compaction requested=true 2023-02-13 16:55:50,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:50,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:50,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:50,409 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:50,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:55:50,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:55:50,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:55:50,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 16:55:50,409 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 16:55:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,411 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,411 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:55:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:50,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:50,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:50,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 16:55:50,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17d6ac02[Count = 0] remaining members to acquire global barrier 2023-02-13 16:55:50,412 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 16:55:50,412 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,413 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 16:55:50,414 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,414 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:55:50,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 16:55:50,414 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:55:50,419 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 16:55:50,419 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,419 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:50,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:50,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:50,419 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 16:55:50,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:50,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:50,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:50,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:50,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:50,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:55:50,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 16:55:50,421 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 16:55:50,421 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:55:50,421 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:55:50,421 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 16:55:50,421 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:55:50,422 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,422 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:55:50,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:55:50,422 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:50,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,423 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:50,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:55:50,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:50,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:50,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:55:50,424 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:50,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:55:50,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 16:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 16:55:50,426 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:55:50,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 16:55:50,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:50,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:55:50,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 16:55:50,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 16:55:50,426 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 16:55:50,526 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:55:50,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:55:50,527 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:50,527 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): c0c2fdbf028f696dc83568e799a88cc4/family1 is initiating major compaction (all files) 2023-02-13 16:55:50,528 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:50,528 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:50,528 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of c0c2fdbf028f696dc83568e799a88cc4/family1 in ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:50,528 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/854b5e1cef26478094a6ff7f1b4c69d1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c6989189d86f48aeb8f4a0d2128333aa, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/9687ad74fa9f46a2973fa4f4eaaca7ef, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c1e8d9e908d148ffbb24e2d2b2d290fb] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp, totalSize=26.6 K 2023-02-13 16:55:50,528 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 854b5e1cef26478094a6ff7f1b4c69d1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307341964 2023-02-13 16:55:50,529 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting c6989189d86f48aeb8f4a0d2128333aa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307343749 2023-02-13 16:55:50,529 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 38ee7eb7f7df4091a4b2fad1ec0fa0ac, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307345544 2023-02-13 16:55:50,529 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9687ad74fa9f46a2973fa4f4eaaca7ef, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307347330 2023-02-13 16:55:50,530 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting c1e8d9e908d148ffbb24e2d2b2d290fb, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:55:50,537 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): c0c2fdbf028f696dc83568e799a88cc4#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:50,945 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302137a415b26dab84384b8147976f5761f19_del 2023-02-13 16:55:50,945 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302137a415b26dab84384b8147976f5761f19_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302137a415b26dab84384b8147976f5761f19_del 2023-02-13 16:55:51,757 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7619918a757644c5b9d1330cb3b9ecc0 2023-02-13 16:55:51,758 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family1/7619918a757644c5b9d1330cb3b9ecc0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/7619918a757644c5b9d1330cb3b9ecc0 2023-02-13 16:55:51,761 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7619918a757644c5b9d1330cb3b9ecc0 2023-02-13 16:55:51,761 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c0c2fdbf028f696dc83568e799a88cc4/family1 of c0c2fdbf028f696dc83568e799a88cc4 into 7619918a757644c5b9d1330cb3b9ecc0(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:51,762 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:51,762 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:51,762 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): c0c2fdbf028f696dc83568e799a88cc4/family2 is initiating major compaction (all files) 2023-02-13 16:55:51,762 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:51,762 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:51,763 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of c0c2fdbf028f696dc83568e799a88cc4/family2 in ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:55:51,763 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/fb87190e155c4e52a4e3cdd74824aefc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/8addf97f1b894062b3de6f897c9f68b4, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/9e07b9a8d70b489aabe00ca8eb2e82cd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/220770ea40ed4868ab853aa5abab3fdc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/a9a44642c5d240a79bbf778c61c3a7ad] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp, totalSize=25.3 K 2023-02-13 16:55:51,763 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting fb87190e155c4e52a4e3cdd74824aefc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307341964 2023-02-13 16:55:51,763 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 8addf97f1b894062b3de6f897c9f68b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307343749 2023-02-13 16:55:51,764 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9e07b9a8d70b489aabe00ca8eb2e82cd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307345544 2023-02-13 16:55:51,764 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 220770ea40ed4868ab853aa5abab3fdc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307347330 2023-02-13 16:55:51,764 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting a9a44642c5d240a79bbf778c61c3a7ad, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:55:51,772 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): c0c2fdbf028f696dc83568e799a88cc4#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:51,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/854b5e1cef26478094a6ff7f1b4c69d1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c6989189d86f48aeb8f4a0d2128333aa, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/9687ad74fa9f46a2973fa4f4eaaca7ef, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c1e8d9e908d148ffbb24e2d2b2d290fb] to archive 2023-02-13 16:55:51,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:51,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/854b5e1cef26478094a6ff7f1b4c69d1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/854b5e1cef26478094a6ff7f1b4c69d1 2023-02-13 16:55:51,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c6989189d86f48aeb8f4a0d2128333aa to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c6989189d86f48aeb8f4a0d2128333aa 2023-02-13 16:55:51,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/38ee7eb7f7df4091a4b2fad1ec0fa0ac 2023-02-13 16:55:51,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/9687ad74fa9f46a2973fa4f4eaaca7ef to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/9687ad74fa9f46a2973fa4f4eaaca7ef 2023-02-13 16:55:51,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c1e8d9e908d148ffbb24e2d2b2d290fb to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/c1e8d9e908d148ffbb24e2d2b2d290fb 2023-02-13 16:55:52,179 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302139db36e92def849bc8fa12d3b27ea52fb_del 2023-02-13 16:55:52,179 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139db36e92def849bc8fa12d3b27ea52fb_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302139db36e92def849bc8fa12d3b27ea52fb_del 2023-02-13 16:55:52,991 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5d522f571c44551ab9c593c5d518375 2023-02-13 16:55:52,992 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/.tmp/family2/f5d522f571c44551ab9c593c5d518375 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/f5d522f571c44551ab9c593c5d518375 2023-02-13 16:55:52,994 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5d522f571c44551ab9c593c5d518375 2023-02-13 16:55:52,995 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c0c2fdbf028f696dc83568e799a88cc4/family2 of c0c2fdbf028f696dc83568e799a88cc4 into f5d522f571c44551ab9c593c5d518375(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:52,995 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:52,995 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:52,996 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): 433223184ab0ee1d5c98c521a6bdfa88/family1 is initiating major compaction (all files) 2023-02-13 16:55:52,996 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:52,996 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:52,996 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of 433223184ab0ee1d5c98c521a6bdfa88/family1 in ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:52,996 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f1b92ef544924afdba942ba8fb28aa02, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/a36f3a4e4f984461b30f85f972615fb9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/430ce9d5cd414df49612aff0db1032fa, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/8eb4eaca2d3741f8a6a805f48be944e6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f4bfa4cc3e8840acab1e9a7689d72057] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp, totalSize=26.6 K 2023-02-13 16:55:52,996 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting f1b92ef544924afdba942ba8fb28aa02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307341962 2023-02-13 16:55:52,996 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting a36f3a4e4f984461b30f85f972615fb9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307343749 2023-02-13 16:55:52,997 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 430ce9d5cd414df49612aff0db1032fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307345544 2023-02-13 16:55:52,997 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 8eb4eaca2d3741f8a6a805f48be944e6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307347330 2023-02-13 16:55:52,997 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting f4bfa4cc3e8840acab1e9a7689d72057, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:55:53,004 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): 433223184ab0ee1d5c98c521a6bdfa88#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:53,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/fb87190e155c4e52a4e3cdd74824aefc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/8addf97f1b894062b3de6f897c9f68b4, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/9e07b9a8d70b489aabe00ca8eb2e82cd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/220770ea40ed4868ab853aa5abab3fdc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/a9a44642c5d240a79bbf778c61c3a7ad] to archive 2023-02-13 16:55:53,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:53,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/fb87190e155c4e52a4e3cdd74824aefc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/fb87190e155c4e52a4e3cdd74824aefc 2023-02-13 16:55:53,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/8addf97f1b894062b3de6f897c9f68b4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/8addf97f1b894062b3de6f897c9f68b4 2023-02-13 16:55:53,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/9e07b9a8d70b489aabe00ca8eb2e82cd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/9e07b9a8d70b489aabe00ca8eb2e82cd 2023-02-13 16:55:53,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/220770ea40ed4868ab853aa5abab3fdc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/220770ea40ed4868ab853aa5abab3fdc 2023-02-13 16:55:53,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/a9a44642c5d240a79bbf778c61c3a7ad to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family2/a9a44642c5d240a79bbf778c61c3a7ad 2023-02-13 16:55:53,412 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021340a2efe09f8e43028bb843f4a35e7b63_del 2023-02-13 16:55:53,412 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021340a2efe09f8e43028bb843f4a35e7b63_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023021340a2efe09f8e43028bb843f4a35e7b63_del 2023-02-13 16:55:54,224 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adad9d5c0b24476ea0ac7b6bc9b3d68b 2023-02-13 16:55:54,225 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family1/adad9d5c0b24476ea0ac7b6bc9b3d68b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/adad9d5c0b24476ea0ac7b6bc9b3d68b 2023-02-13 16:55:54,228 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adad9d5c0b24476ea0ac7b6bc9b3d68b 2023-02-13 16:55:54,229 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 433223184ab0ee1d5c98c521a6bdfa88/family1 of 433223184ab0ee1d5c98c521a6bdfa88 into adad9d5c0b24476ea0ac7b6bc9b3d68b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:54,229 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:54,229 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:54,229 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): 433223184ab0ee1d5c98c521a6bdfa88/family2 is initiating major compaction (all files) 2023-02-13 16:55:54,230 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:54,230 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:54,230 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of 433223184ab0ee1d5c98c521a6bdfa88/family2 in ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:55:54,230 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/2ab871df26f144e59e770f8a6e0428e3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/f94258edd9b9401e9175fd5dcbcf9dc7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/b5518e29ae90443bb7a5d1b84788a0d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/649fb3f2ef51455189e7b98a3549ea0b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/958fae319e0a4b029e9576523ab7e8ce] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp, totalSize=25.3 K 2023-02-13 16:55:54,230 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 2ab871df26f144e59e770f8a6e0428e3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307341962 2023-02-13 16:55:54,230 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting f94258edd9b9401e9175fd5dcbcf9dc7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307343749 2023-02-13 16:55:54,231 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting b5518e29ae90443bb7a5d1b84788a0d6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307345544 2023-02-13 16:55:54,231 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 649fb3f2ef51455189e7b98a3549ea0b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307347330 2023-02-13 16:55:54,231 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 958fae319e0a4b029e9576523ab7e8ce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:55:54,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f1b92ef544924afdba942ba8fb28aa02, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/a36f3a4e4f984461b30f85f972615fb9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/430ce9d5cd414df49612aff0db1032fa, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/8eb4eaca2d3741f8a6a805f48be944e6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f4bfa4cc3e8840acab1e9a7689d72057] to archive 2023-02-13 16:55:54,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:54,240 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): 433223184ab0ee1d5c98c521a6bdfa88#family2#compaction#152 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:54,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f1b92ef544924afdba942ba8fb28aa02 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f1b92ef544924afdba942ba8fb28aa02 2023-02-13 16:55:54,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/a36f3a4e4f984461b30f85f972615fb9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/a36f3a4e4f984461b30f85f972615fb9 2023-02-13 16:55:54,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/430ce9d5cd414df49612aff0db1032fa to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/430ce9d5cd414df49612aff0db1032fa 2023-02-13 16:55:54,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/8eb4eaca2d3741f8a6a805f48be944e6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/8eb4eaca2d3741f8a6a805f48be944e6 2023-02-13 16:55:54,245 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f4bfa4cc3e8840acab1e9a7689d72057 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/f4bfa4cc3e8840acab1e9a7689d72057 2023-02-13 16:55:54,649 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021363c758356f454ab6ab36a584b5fd7457_del 2023-02-13 16:55:54,650 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021363c758356f454ab6ab36a584b5fd7457_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023021363c758356f454ab6ab36a584b5fd7457_del 2023-02-13 16:55:55,462 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d97f9e48ae3247748d2be2bc64406d2f 2023-02-13 16:55:55,463 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/.tmp/family2/d97f9e48ae3247748d2be2bc64406d2f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/d97f9e48ae3247748d2be2bc64406d2f 2023-02-13 16:55:55,466 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d97f9e48ae3247748d2be2bc64406d2f 2023-02-13 16:55:55,467 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 433223184ab0ee1d5c98c521a6bdfa88/family2 of 433223184ab0ee1d5c98c521a6bdfa88 into d97f9e48ae3247748d2be2bc64406d2f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:55,467 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:55,467 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:55,467 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): d345489d398503e06fb04841004d8255/family1 is initiating major compaction (all files) 2023-02-13 16:55:55,467 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:55,467 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:55,468 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of d345489d398503e06fb04841004d8255/family1 in ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:55,468 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/e9471fa873754d5882950adf18670105, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/93d8322480b745beb13ca54b670f86d1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/9d0dc7420d534f48afdcf7c64ce965d7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/56051bd0c56441df88506d3a20580da8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/ea750dc704514772863469874a3147da] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp, totalSize=26.6 K 2023-02-13 16:55:55,469 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting e9471fa873754d5882950adf18670105, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307341964 2023-02-13 16:55:55,469 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 93d8322480b745beb13ca54b670f86d1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307343749 2023-02-13 16:55:55,469 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 9d0dc7420d534f48afdcf7c64ce965d7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307345545 2023-02-13 16:55:55,469 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 56051bd0c56441df88506d3a20580da8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307347330 2023-02-13 16:55:55,470 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting ea750dc704514772863469874a3147da, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:55:55,477 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): d345489d398503e06fb04841004d8255#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:55,544 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/2ab871df26f144e59e770f8a6e0428e3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/f94258edd9b9401e9175fd5dcbcf9dc7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/b5518e29ae90443bb7a5d1b84788a0d6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/649fb3f2ef51455189e7b98a3549ea0b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/958fae319e0a4b029e9576523ab7e8ce] to archive 2023-02-13 16:55:55,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:55,546 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/2ab871df26f144e59e770f8a6e0428e3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/2ab871df26f144e59e770f8a6e0428e3 2023-02-13 16:55:55,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/f94258edd9b9401e9175fd5dcbcf9dc7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/f94258edd9b9401e9175fd5dcbcf9dc7 2023-02-13 16:55:55,548 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/b5518e29ae90443bb7a5d1b84788a0d6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/b5518e29ae90443bb7a5d1b84788a0d6 2023-02-13 16:55:55,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/649fb3f2ef51455189e7b98a3549ea0b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/649fb3f2ef51455189e7b98a3549ea0b 2023-02-13 16:55:55,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/958fae319e0a4b029e9576523ab7e8ce to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family2/958fae319e0a4b029e9576523ab7e8ce 2023-02-13 16:55:55,884 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302133691cfd1412c48df9a52bf5fcdfa7096_del 2023-02-13 16:55:55,884 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302133691cfd1412c48df9a52bf5fcdfa7096_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302133691cfd1412c48df9a52bf5fcdfa7096_del 2023-02-13 16:55:56,696 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 230b4b10131a4ce7a0d982aad255e749 2023-02-13 16:55:56,697 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family1/230b4b10131a4ce7a0d982aad255e749 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/230b4b10131a4ce7a0d982aad255e749 2023-02-13 16:55:56,700 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 230b4b10131a4ce7a0d982aad255e749 2023-02-13 16:55:56,701 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d345489d398503e06fb04841004d8255/family1 of d345489d398503e06fb04841004d8255 into 230b4b10131a4ce7a0d982aad255e749(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:56,701 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:56,701 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:55:56,701 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): d345489d398503e06fb04841004d8255/family2 is initiating major compaction (all files) 2023-02-13 16:55:56,701 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:55:56,701 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:55:56,702 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of d345489d398503e06fb04841004d8255/family2 in ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:55:56,703 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/086860fe730147e2bb5b46e3af260fd6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/16b29f83ffce4cce92c579fcc6db5b8f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/f02fb546b823405fa27d3aa8238612a5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/2e6f9dc9421c4be183b4afa24421b027, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/b3ff07e63d33489aa1d9072c7a541450] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp, totalSize=25.3 K 2023-02-13 16:55:56,703 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 086860fe730147e2bb5b46e3af260fd6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307341964 2023-02-13 16:55:56,703 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 16b29f83ffce4cce92c579fcc6db5b8f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307343749 2023-02-13 16:55:56,703 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting f02fb546b823405fa27d3aa8238612a5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307345545 2023-02-13 16:55:56,704 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 2e6f9dc9421c4be183b4afa24421b027, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307347330 2023-02-13 16:55:56,704 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting b3ff07e63d33489aa1d9072c7a541450, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:55:56,713 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): d345489d398503e06fb04841004d8255#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:55:56,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/e9471fa873754d5882950adf18670105, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/93d8322480b745beb13ca54b670f86d1, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/9d0dc7420d534f48afdcf7c64ce965d7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/56051bd0c56441df88506d3a20580da8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/ea750dc704514772863469874a3147da] to archive 2023-02-13 16:55:56,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:56,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/e9471fa873754d5882950adf18670105 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/e9471fa873754d5882950adf18670105 2023-02-13 16:55:56,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/93d8322480b745beb13ca54b670f86d1 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/93d8322480b745beb13ca54b670f86d1 2023-02-13 16:55:56,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/9d0dc7420d534f48afdcf7c64ce965d7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/9d0dc7420d534f48afdcf7c64ce965d7 2023-02-13 16:55:56,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/56051bd0c56441df88506d3a20580da8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/56051bd0c56441df88506d3a20580da8 2023-02-13 16:55:56,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/ea750dc704514772863469874a3147da to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/ea750dc704514772863469874a3147da 2023-02-13 16:55:57,125 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213780eae5527c844fa95a09346acdedd98_del 2023-02-13 16:55:57,125 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213780eae5527c844fa95a09346acdedd98_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213780eae5527c844fa95a09346acdedd98_del 2023-02-13 16:55:57,939 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7be9206b2328431b916d9213344bb562 2023-02-13 16:55:57,940 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/.tmp/family2/7be9206b2328431b916d9213344bb562 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/7be9206b2328431b916d9213344bb562 2023-02-13 16:55:57,943 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7be9206b2328431b916d9213344bb562 2023-02-13 16:55:57,944 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d345489d398503e06fb04841004d8255/family2 of d345489d398503e06fb04841004d8255 into 7be9206b2328431b916d9213344bb562(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:55:57,944 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:57,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/086860fe730147e2bb5b46e3af260fd6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/16b29f83ffce4cce92c579fcc6db5b8f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/f02fb546b823405fa27d3aa8238612a5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/2e6f9dc9421c4be183b4afa24421b027, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/b3ff07e63d33489aa1d9072c7a541450] to archive 2023-02-13 16:55:57,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:57,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/086860fe730147e2bb5b46e3af260fd6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/086860fe730147e2bb5b46e3af260fd6 2023-02-13 16:55:57,949 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:57,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/16b29f83ffce4cce92c579fcc6db5b8f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/16b29f83ffce4cce92c579fcc6db5b8f 2023-02-13 16:55:57,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/f02fb546b823405fa27d3aa8238612a5 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/f02fb546b823405fa27d3aa8238612a5 2023-02-13 16:55:57,952 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:57,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/2e6f9dc9421c4be183b4afa24421b027 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/2e6f9dc9421c4be183b4afa24421b027 2023-02-13 16:55:57,956 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:57,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/b3ff07e63d33489aa1d9072c7a541450 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family2/b3ff07e63d33489aa1d9072c7a541450 2023-02-13 16:55:57,957 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:57,965 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 16:55:57,979 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-13 16:55:57,982 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021340a2efe09f8e43028bb843f4a35e7b63_del 2023-02-13 16:55:57,985 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302137a415b26dab84384b8147976f5761f19_del 2023-02-13 16:55:57,988 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302133691cfd1412c48df9a52bf5fcdfa7096_del 2023-02-13 16:55:57,988 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:57,989 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(139): Connect 0x1a0f9b20 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:55:57,998 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f6c2fc6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:55:57,999 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 16:55:57,999 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 16:55:57,999 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 16:55:58,054 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213afbd219cf250410195eb49af74733637 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213afbd219cf250410195eb49af74733637 2023-02-13 16:55:58,055 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,073 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:55:58,076 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50728, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:55:58,086 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:55:58,091 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37272, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:55:58,111 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/1f417efda2eb49619430ef5e0f6dd7de first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-13 16:55:58,112 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/1f417efda2eb49619430ef5e0f6dd7de}] 2023-02-13 16:55:58,124 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/1f417efda2eb49619430ef5e0f6dd7de for inclusion in 433223184ab0ee1d5c98c521a6bdfa88/family1 2023-02-13 16:55:58,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-13 16:55:58,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 16:55:58,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for 433223184ab0ee1d5c98c521a6bdfa88: 2023-02-13 16:55:58,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/1f417efda2eb49619430ef5e0f6dd7de to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__q3b865lrtauaa2f520kecf20etj94jrnanpaiaeckbha000jt3lco3c71583qfg2/family1/1f417efda2eb49619430ef5e0f6dd7de 2023-02-13 16:55:58,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__q3b865lrtauaa2f520kecf20etj94jrnanpaiaeckbha000jt3lco3c71583qfg2/family1/1f417efda2eb49619430ef5e0f6dd7de as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/cca31f2e730e48529d5856d350fa3746_SeqId_28_ 2023-02-13 16:55:58,130 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__q3b865lrtauaa2f520kecf20etj94jrnanpaiaeckbha000jt3lco3c71583qfg2/family1/1f417efda2eb49619430ef5e0f6dd7de into 433223184ab0ee1d5c98c521a6bdfa88/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/cca31f2e730e48529d5856d350fa3746_SeqId_28_ - updating store file list. 2023-02-13 16:55:58,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/cca31f2e730e48529d5856d350fa3746_SeqId_28_ into 433223184ab0ee1d5c98c521a6bdfa88/family1 2023-02-13 16:55:58,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__q3b865lrtauaa2f520kecf20etj94jrnanpaiaeckbha000jt3lco3c71583qfg2/family1/1f417efda2eb49619430ef5e0f6dd7de into 433223184ab0ee1d5c98c521a6bdfa88/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/family1/cca31f2e730e48529d5856d350fa3746_SeqId_28_) 2023-02-13 16:55:58,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__q3b865lrtauaa2f520kecf20etj94jrnanpaiaeckbha000jt3lco3c71583qfg2/family1/1f417efda2eb49619430ef5e0f6dd7de 2023-02-13 16:55:58,135 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,136 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:58,137 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213b568f899bf244fa29799bc639e0c2a31 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213b568f899bf244fa29799bc639e0c2a31 2023-02-13 16:55:58,137 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 16:55:58,137 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 16:55:58,137 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 16:55:58,453 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021375501afdbfae40fe832f8c6fab0e9487 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021375501afdbfae40fe832f8c6fab0e9487 2023-02-13 16:55:58,454 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,456 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f3e851502fb3461c9823fa04e64ebe2e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213f3e851502fb3461c9823fa04e64ebe2e 2023-02-13 16:55:58,457 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,478 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/ed3aed2df2274be7ac0c1782f9b179d7 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-13 16:55:58,478 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/ed3aed2df2274be7ac0c1782f9b179d7}] 2023-02-13 16:55:58,479 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/df186f766c5a4f9caa9d7f3aff0783cd first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-13 16:55:58,480 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/df186f766c5a4f9caa9d7f3aff0783cd}] 2023-02-13 16:55:58,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/ed3aed2df2274be7ac0c1782f9b179d7 for inclusion in d345489d398503e06fb04841004d8255/family1 2023-02-13 16:55:58,482 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/df186f766c5a4f9caa9d7f3aff0783cd for inclusion in c0c2fdbf028f696dc83568e799a88cc4/family1 2023-02-13 16:55:58,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-13 16:55:58,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 16:55:58,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for d345489d398503e06fb04841004d8255: 2023-02-13 16:55:58,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-13 16:55:58,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 16:55:58,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for c0c2fdbf028f696dc83568e799a88cc4: 2023-02-13 16:55:58,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/ed3aed2df2274be7ac0c1782f9b179d7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__4hecegl5je9ofebnlrqqcf36276r5fehsrnlvo6njusvmq1lkpg7ct9s2q8pj3k1/family1/ed3aed2df2274be7ac0c1782f9b179d7 2023-02-13 16:55:58,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/df186f766c5a4f9caa9d7f3aff0783cd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__7sph8pk8u1jhj319ccfarbr3iehqlvbubtg456qqdad4inh1d8e0rhbsa0bc6vdd/family1/df186f766c5a4f9caa9d7f3aff0783cd 2023-02-13 16:55:58,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__4hecegl5je9ofebnlrqqcf36276r5fehsrnlvo6njusvmq1lkpg7ct9s2q8pj3k1/family1/ed3aed2df2274be7ac0c1782f9b179d7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/4508df1b661d440d824007863f944c3d_SeqId_28_ 2023-02-13 16:55:58,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__7sph8pk8u1jhj319ccfarbr3iehqlvbubtg456qqdad4inh1d8e0rhbsa0bc6vdd/family1/df186f766c5a4f9caa9d7f3aff0783cd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/adab7404320c4decbf6c83a3aabc65a6_SeqId_28_ 2023-02-13 16:55:58,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__4hecegl5je9ofebnlrqqcf36276r5fehsrnlvo6njusvmq1lkpg7ct9s2q8pj3k1/family1/ed3aed2df2274be7ac0c1782f9b179d7 into d345489d398503e06fb04841004d8255/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/4508df1b661d440d824007863f944c3d_SeqId_28_ - updating store file list. 2023-02-13 16:55:58,487 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__7sph8pk8u1jhj319ccfarbr3iehqlvbubtg456qqdad4inh1d8e0rhbsa0bc6vdd/family1/df186f766c5a4f9caa9d7f3aff0783cd into c0c2fdbf028f696dc83568e799a88cc4/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/adab7404320c4decbf6c83a3aabc65a6_SeqId_28_ - updating store file list. 2023-02-13 16:55:58,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/4508df1b661d440d824007863f944c3d_SeqId_28_ into d345489d398503e06fb04841004d8255/family1 2023-02-13 16:55:58,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__4hecegl5je9ofebnlrqqcf36276r5fehsrnlvo6njusvmq1lkpg7ct9s2q8pj3k1/family1/ed3aed2df2274be7ac0c1782f9b179d7 into d345489d398503e06fb04841004d8255/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/family1/4508df1b661d440d824007863f944c3d_SeqId_28_) 2023-02-13 16:55:58,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/adab7404320c4decbf6c83a3aabc65a6_SeqId_28_ into c0c2fdbf028f696dc83568e799a88cc4/family1 2023-02-13 16:55:58,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__7sph8pk8u1jhj319ccfarbr3iehqlvbubtg456qqdad4inh1d8e0rhbsa0bc6vdd/family1/df186f766c5a4f9caa9d7f3aff0783cd into c0c2fdbf028f696dc83568e799a88cc4/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/family1/adab7404320c4decbf6c83a3aabc65a6_SeqId_28_) 2023-02-13 16:55:58,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__4hecegl5je9ofebnlrqqcf36276r5fehsrnlvo6njusvmq1lkpg7ct9s2q8pj3k1/family1/ed3aed2df2274be7ac0c1782f9b179d7 2023-02-13 16:55:58,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__ns_testMinorCompaction__7sph8pk8u1jhj319ccfarbr3iehqlvbubtg456qqdad4inh1d8e0rhbsa0bc6vdd/family1/df186f766c5a4f9caa9d7f3aff0783cd 2023-02-13 16:55:58,493 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,493 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,493 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:58,493 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:55:58,495 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021366aba80442274460a4831d316f87ca1b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021366aba80442274460a4831d316f87ca1b 2023-02-13 16:55:58,495 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302138703c6257f3b431bb2d2752910a17db2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302138703c6257f3b431bb2d2752910a17db2 2023-02-13 16:55:58,495 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 16:55:58,495 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 16:55:58,495 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 16:55:58,495 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 16:55:58,495 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 16:55:58,495 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 16:55:58,495 INFO [Listener at localhost/36833] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-13 16:55:58,497 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:58,499 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:58,501 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:55:58,502 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:55:58,517 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=246 (was 286), OpenFileDescriptor=630 (was 765), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=117 (was 134), ProcessCount=168 (was 168), AvailableMemoryMB=1104 (was 1113) 2023-02-13 16:55:58,527 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=246, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=117, ProcessCount=168, AvailableMemoryMB=1103 2023-02-13 16:55:58,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 16:55:58,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-13 16:55:58,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:55:58,531 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-13 16:55:58,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:55:58,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 16:55:58,533 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:58,533 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73 empty. 2023-02-13 16:55:58,533 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:58,533 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-13 16:55:58,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 16:55:58,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 16:55:58,939 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 16:55:58,940 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => f662746060867d63effc2a9ff57f8b73, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:55:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 16:55:59,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:59,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing f662746060867d63effc2a9ff57f8b73, disabling compactions & flushes 2023-02-13 16:55:59,346 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. after waiting 0 ms 2023-02-13 16:55:59,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,346 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for f662746060867d63effc2a9ff57f8b73: Waiting for close lock at 1676307359346Disabling compacts and flushes for region at 1676307359346Disabling writes for close at 1676307359346Writing region close event to WAL at 1676307359346Closed at 1676307359346 2023-02-13 16:55:59,349 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:55:59,350 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307359349"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307359349"}]},"ts":"1676307359349"} 2023-02-13 16:55:59,351 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 16:55:59,352 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:55:59,352 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307359352"}]},"ts":"1676307359352"} 2023-02-13 16:55:59,353 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-13 16:55:59,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f662746060867d63effc2a9ff57f8b73, ASSIGN}] 2023-02-13 16:55:59,358 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f662746060867d63effc2a9ff57f8b73, ASSIGN 2023-02-13 16:55:59,358 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f662746060867d63effc2a9ff57f8b73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:55:59,509 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=f662746060867d63effc2a9ff57f8b73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:59,509 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307359509"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307359509"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307359509"}]},"ts":"1676307359509"} 2023-02-13 16:55:59,511 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure f662746060867d63effc2a9ff57f8b73, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:55:59,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 16:55:59,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f662746060867d63effc2a9ff57f8b73, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.', STARTKEY => '', ENDKEY => ''} 2023-02-13 16:55:59,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:55:59,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,666 INFO [StoreOpener-f662746060867d63effc2a9ff57f8b73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,667 DEBUG [StoreOpener-f662746060867d63effc2a9ff57f8b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:55:59,667 DEBUG [StoreOpener-f662746060867d63effc2a9ff57f8b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:55:59,667 INFO [StoreOpener-f662746060867d63effc2a9ff57f8b73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f662746060867d63effc2a9ff57f8b73 columnFamilyName family1 2023-02-13 16:55:59,668 INFO [StoreOpener-f662746060867d63effc2a9ff57f8b73-1] regionserver.HStore(310): Store=f662746060867d63effc2a9ff57f8b73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:55:59,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f662746060867d63effc2a9ff57f8b73 2023-02-13 16:55:59,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:55:59,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f662746060867d63effc2a9ff57f8b73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63089780, jitterRate=-0.059889018535614014}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 16:55:59,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f662746060867d63effc2a9ff57f8b73: Running coprocessor pre-open hook at 1676307359665Writing region info on filesystem at 1676307359665Initializing all the Stores at 1676307359665Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676307359665Cleaning up temporary data from old regions at 1676307359669 (+4 ms)Cleaning up detritus from prior splits at 1676307359670 (+1 ms)Running coprocessor post-open hooks at 1676307359672 (+2 ms)Region opened successfully at 1676307359672 2023-02-13 16:55:59,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73., pid=114, masterSystemTime=1676307359662 2023-02-13 16:55:59,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:55:59,674 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=f662746060867d63effc2a9ff57f8b73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:55:59,674 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676307359674"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307359674"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307359674"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307359674"}]},"ts":"1676307359674"} 2023-02-13 16:55:59,676 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 16:55:59,677 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure f662746060867d63effc2a9ff57f8b73, server=jenkins-hbase4.apache.org,33907,1676307219708 in 164 msec 2023-02-13 16:55:59,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 16:55:59,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f662746060867d63effc2a9ff57f8b73, ASSIGN in 321 msec 2023-02-13 16:55:59,678 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:55:59,678 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307359678"}]},"ts":"1676307359678"} 2023-02-13 16:55:59,679 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-13 16:55:59,681 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:55:59,682 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1520 sec 2023-02-13 16:56:00,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 16:56:00,636 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-13 16:56:00,651 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:00,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,653 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:00,653 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,653 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:00,653 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:00,653 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,653 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,655 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:00,655 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:00,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:00,656 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:00,656 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:00,657 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:00,657 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:00,657 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:00,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:00,658 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:00,658 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:00,658 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:00,659 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 16:56:01,079 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130515cfdc2b1994626803105817c98e8df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130515cfdc2b1994626803105817c98e8df 2023-02-13 16:56:01,482 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/c94d0e8470094af0bcdaad42947c83fe 2023-02-13 16:56:01,486 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/c94d0e8470094af0bcdaad42947c83fe as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c94d0e8470094af0bcdaad42947c83fe 2023-02-13 16:56:01,489 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c94d0e8470094af0bcdaad42947c83fe, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-13 16:56:01,490 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 832ms, sequenceid=5, compaction requested=false 2023-02-13 16:56:01,490 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:01,491 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:01,491 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:01,491 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:01,491 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:01,491 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,491 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:01,491 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:01,492 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,493 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:01,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d0b88d[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:01,495 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:01,495 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,496 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:01,496 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,496 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:01,498 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,498 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:01,498 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,498 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:01,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:01,501 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:01,501 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:01,502 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:01,502 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,502 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:01,504 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,504 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:01,504 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,504 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:01,505 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:01,507 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,507 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,507 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:01,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,508 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:01,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:01,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,508 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:01,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:01,608 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:01,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:01,613 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:01,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:01,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,614 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:01,614 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,615 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:01,615 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:01,615 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,615 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,616 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:01,616 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:01,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:01,617 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:01,617 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:01,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:01,618 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:01,618 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:01,618 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:01,618 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:01,618 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:01,618 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:01,618 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:01,618 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 16:56:02,030 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112859306fd57a5d48c29e39a84b915b9039 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112859306fd57a5d48c29e39a84b915b9039 2023-02-13 16:56:02,034 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/c2765a67910445cf893236d91dd63ce5 2023-02-13 16:56:02,037 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/c2765a67910445cf893236d91dd63ce5 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c2765a67910445cf893236d91dd63ce5 2023-02-13 16:56:02,040 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c2765a67910445cf893236d91dd63ce5, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 16:56:02,041 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 423ms, sequenceid=9, compaction requested=false 2023-02-13 16:56:02,041 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:02,041 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:02,041 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:02,041 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:02,042 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,042 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:02,042 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:02,043 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,043 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:02,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:02,043 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,043 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:02,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@295b6722[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:02,045 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:02,045 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,046 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,046 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:02,046 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:02,046 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:02,046 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,046 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:02,048 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,048 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:02,048 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:02,048 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:02,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:02,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:02,050 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:02,050 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:02,050 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:02,050 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,050 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:02,052 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,052 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:02,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:02,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,052 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:02,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:02,053 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:02,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:02,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:02,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,053 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:02,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,055 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,055 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:02,055 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,055 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:02,055 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:02,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:02,055 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,055 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:02,056 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:02,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:02,055 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:02,056 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,056 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:02,056 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,056 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:02,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:02,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:02,056 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,156 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:02,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:02,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:02,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:02,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,162 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:02,162 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:02,162 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:02,162 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:02,162 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,162 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,171 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,171 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:02,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:02,172 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,172 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,173 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:02,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,173 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:02,173 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:02,173 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:02,173 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:02,173 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:02,173 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:02,173 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:02,173 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:02,583 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015b070fc9da664020929030e1752e8fd4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512015b070fc9da664020929030e1752e8fd4 2023-02-13 16:56:02,988 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/f7902c3a9ef04649b5edac4d5f43a929 2023-02-13 16:56:02,992 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/f7902c3a9ef04649b5edac4d5f43a929 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/f7902c3a9ef04649b5edac4d5f43a929 2023-02-13 16:56:02,995 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/f7902c3a9ef04649b5edac4d5f43a929, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 16:56:02,996 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 823ms, sequenceid=13, compaction requested=true 2023-02-13 16:56:02,996 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:02,996 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:02,996 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:02,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:02,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:02,997 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:02,999 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:02,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:02,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:02,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,000 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:03,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:03,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:03,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:03,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:03,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21a6a39b[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:03,001 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:03,001 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,002 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,003 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:03,003 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:03,003 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,003 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:03,003 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:03,004 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:03,004 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,004 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:03,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:03,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:03,004 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:03,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:03,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:03,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:03,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:03,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:03,007 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:03,007 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:03,007 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:03,007 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,007 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:03,010 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,010 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:03,010 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:03,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:03,010 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,011 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:03,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,016 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,016 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:03,016 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,016 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:03,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:03,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:03,016 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,016 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,017 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:03,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:03,016 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:03,017 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:03,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:03,017 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,017 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:03,017 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:03,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:03,117 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:03,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:03,125 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:03,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:03,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,127 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:03,127 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:03,127 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:03,127 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:03,128 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,128 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,129 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:03,130 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,130 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,130 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:03,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:03,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:03,131 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:03,131 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:03,131 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:03,545 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205470379edc13a4a9ab0e94650b723b423 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205470379edc13a4a9ab0e94650b723b423 2023-02-13 16:56:03,950 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/9056403cdb90485c8505e6559a0c2ee8 2023-02-13 16:56:03,955 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/9056403cdb90485c8505e6559a0c2ee8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9056403cdb90485c8505e6559a0c2ee8 2023-02-13 16:56:03,959 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9056403cdb90485c8505e6559a0c2ee8, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 16:56:03,960 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 829ms, sequenceid=17, compaction requested=true 2023-02-13 16:56:03,961 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:03,961 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:03,961 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:03,961 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:03,961 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,961 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:03,961 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:03,963 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,963 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:03,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:03,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,964 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:03,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:03,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:03,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:03,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:03,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b2ad11[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:03,965 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:03,966 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,967 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,967 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:03,967 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:03,967 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:03,967 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,967 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:03,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:03,969 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:03,969 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:03,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:03,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:03,972 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:03,972 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:03,972 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:03,972 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,972 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:03,975 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,975 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:03,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,975 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:03,975 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,976 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:03,979 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,979 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:03,979 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:03,979 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:03,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:03,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:03,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:03,980 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:03,980 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:03,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:03,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:04,080 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:04,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:04,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:04,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:04,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,085 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:04,085 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:04,085 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:04,085 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:04,086 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,086 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,088 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,088 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:04,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:04,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:04,088 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,088 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:04,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:04,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:04,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:04,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:04,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:04,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:04,089 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:04,089 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:04,089 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:04,089 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:04,500 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228889e6067716247aeb3c85691ab5c64a4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228889e6067716247aeb3c85691ab5c64a4 2023-02-13 16:56:04,904 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/5c762176de874ca4b03d52568ef88558 2023-02-13 16:56:04,910 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/5c762176de874ca4b03d52568ef88558 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/5c762176de874ca4b03d52568ef88558 2023-02-13 16:56:04,913 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/5c762176de874ca4b03d52568ef88558, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 16:56:04,914 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 825ms, sequenceid=21, compaction requested=true 2023-02-13 16:56:04,914 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:04,914 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:04,914 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:04,914 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:04,914 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,914 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:04,914 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:04,917 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,917 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:04,917 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,918 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:04,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:04,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:04,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:04,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:04,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a3793a[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:04,919 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:04,919 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,920 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,920 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:04,920 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:04,920 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:04,920 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,920 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:04,921 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,921 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:04,921 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:04,921 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:04,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:04,923 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:04,923 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:04,923 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:04,923 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,923 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:04,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:04,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,925 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:04,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:04,926 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:04,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:04,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:04,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,927 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:04,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,933 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,933 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:04,933 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:04,933 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:04,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:04,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:04,933 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:04,933 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:04,933 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:04,934 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,934 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,934 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:04,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:04,934 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:04,934 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:05,034 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:05,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:05,038 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:05,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:05,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,040 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:05,040 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:05,040 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:05,040 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:05,040 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,040 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,041 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:05,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:05,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:05,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,041 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:05,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:05,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,042 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:05,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,042 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:05,042 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:05,042 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:05,043 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:05,043 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:05,043 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:05,043 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:05,043 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:05,203 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:56:05,453 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231eef38173a1cc4c4da990de5ff851a689 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231eef38173a1cc4c4da990de5ff851a689 2023-02-13 16:56:05,857 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/61ac595e258846f0ba64205cdcf60007 2023-02-13 16:56:05,861 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/61ac595e258846f0ba64205cdcf60007 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/61ac595e258846f0ba64205cdcf60007 2023-02-13 16:56:05,864 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/61ac595e258846f0ba64205cdcf60007, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 16:56:05,865 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 822ms, sequenceid=25, compaction requested=true 2023-02-13 16:56:05,865 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:05,865 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:05,865 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:05,865 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:05,865 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,865 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:05,865 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:05,866 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,866 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:05,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:05,867 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,867 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:05,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:05,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:05,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:05,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:05,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74337cbb[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:05,868 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:05,868 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,870 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,870 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:05,870 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,870 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:05,870 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:05,870 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:05,871 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,871 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:05,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:05,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:05,871 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:05,872 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:05,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:05,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:05,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:05,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:05,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:05,874 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:05,874 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:05,874 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:05,874 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,874 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:05,875 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,875 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:05,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:05,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,875 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:05,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:05,876 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:05,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:05,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:05,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,877 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:05,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:05,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:05,879 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:05,879 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:05,879 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:05,879 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:05,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:05,980 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:05,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:05,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:05,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:05,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,985 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:05,985 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:05,985 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:05,985 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:05,986 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,986 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,992 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,992 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:05,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:05,992 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:05,992 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:05,993 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:05,993 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:05,993 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:05,994 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:05,994 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:05,994 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:06,405 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601013d2be505248d452facd3df9c4413e333 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601013d2be505248d452facd3df9c4413e333 2023-02-13 16:56:06,809 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/9d6f62539b3b41aaa4fdf01e1eb7b230 2023-02-13 16:56:06,813 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/9d6f62539b3b41aaa4fdf01e1eb7b230 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9d6f62539b3b41aaa4fdf01e1eb7b230 2023-02-13 16:56:06,815 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9d6f62539b3b41aaa4fdf01e1eb7b230, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 16:56:06,816 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 822ms, sequenceid=29, compaction requested=true 2023-02-13 16:56:06,816 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:06,816 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:06,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:06,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:06,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:06,816 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:06,818 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:06,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:06,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,818 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:06,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:06,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:06,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:06,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:06,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@609db3dd[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:06,819 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:06,819 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,820 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:06,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:06,820 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,820 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:06,820 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:06,821 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:06,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:06,822 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:06,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:06,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:06,823 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:06,823 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:06,823 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:06,823 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,824 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:06,826 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,826 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:06,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:06,826 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:06,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,827 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:06,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,828 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,829 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,830 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:06,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:06,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:06,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:06,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:06,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:06,830 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:06,930 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:06,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:06,934 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:06,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:06,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,936 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:06,936 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:06,936 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:06,936 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:06,936 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,936 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,937 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:06,937 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:06,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:06,938 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:06,938 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:06,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:06,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:06,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:06,939 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:06,939 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:06,939 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:06,939 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:06,939 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:06,939 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:06,939 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:06,939 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:07,349 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103250c8889d58a40b4ac0af411462f59bf to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103250c8889d58a40b4ac0af411462f59bf 2023-02-13 16:56:07,753 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/662147028b304acab281ea053671ad24 2023-02-13 16:56:07,757 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/662147028b304acab281ea053671ad24 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/662147028b304acab281ea053671ad24 2023-02-13 16:56:07,761 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/662147028b304acab281ea053671ad24, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 16:56:07,762 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 823ms, sequenceid=33, compaction requested=true 2023-02-13 16:56:07,762 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:07,762 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:07,762 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:07,762 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:07,762 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,762 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:07,762 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:07,763 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,763 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,764 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,764 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:07,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:07,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@765140fa[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:07,765 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:07,765 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,766 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,767 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:07,767 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:07,767 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,767 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:07,767 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:07,771 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:07,771 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,771 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,771 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:07,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:07,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:07,773 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:07,773 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:07,773 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:07,773 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,773 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:07,775 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,775 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,775 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:07,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,775 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:07,776 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:07,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,781 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:07,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:07,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:07,781 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:07,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:07,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:07,882 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:07,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:07,886 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:07,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:07,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,887 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:07,887 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,887 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:07,887 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:07,888 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,888 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,889 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,889 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:07,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:07,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:07,889 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:07,889 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:07,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:07,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,890 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:07,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:07,890 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:07,890 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:07,890 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:08,305 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f94ea175dcd44ce1a7649de5f6dd9007 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907f94ea175dcd44ce1a7649de5f6dd9007 2023-02-13 16:56:08,727 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/b24357a9fd604b6ca38c6db36bba8551 2023-02-13 16:56:08,731 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/b24357a9fd604b6ca38c6db36bba8551 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/b24357a9fd604b6ca38c6db36bba8551 2023-02-13 16:56:08,735 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/b24357a9fd604b6ca38c6db36bba8551, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 16:56:08,736 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 846ms, sequenceid=37, compaction requested=true 2023-02-13 16:56:08,736 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:08,736 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:08,738 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,738 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:08,738 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,738 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ba68da3[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:08,739 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:08,740 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,741 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,741 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:08,741 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:08,741 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:08,741 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,741 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:08,743 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,743 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:08,743 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:08,743 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:08,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:08,746 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:08,746 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:08,746 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:08,746 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,746 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:08,748 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:08,748 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:08,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,748 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:08,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:08,748 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,751 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:08,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,753 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:08,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:08,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:08,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:08,753 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:08,754 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:08,754 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,754 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:08,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:08,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:08,854 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:08,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:08,858 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:08,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:08,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,860 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:08,860 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:08,860 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 16:56:08,860 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 16:56:08,861 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,861 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,862 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,862 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:08,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:08,862 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:08,862 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,862 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,863 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,863 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 16:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 16:56:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 16:56:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 16:56:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:08,863 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:08,863 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. started... 2023-02-13 16:56:08,863 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f662746060867d63effc2a9ff57f8b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 16:56:09,278 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ed36f2c1b8744149bffb936e4227234c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120ed36f2c1b8744149bffb936e4227234c 2023-02-13 16:56:09,283 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/26aeda4f1ba0434a80d70d905453511c 2023-02-13 16:56:09,286 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/26aeda4f1ba0434a80d70d905453511c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/26aeda4f1ba0434a80d70d905453511c 2023-02-13 16:56:09,289 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/26aeda4f1ba0434a80d70d905453511c, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 16:56:09,290 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f662746060867d63effc2a9ff57f8b73 in 427ms, sequenceid=41, compaction requested=true 2023-02-13 16:56:09,290 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:09,290 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:09,290 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 16:56:09,290 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 16:56:09,290 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,290 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 16:56:09,290 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 16:56:09,293 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,293 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:09,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:09,293 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,293 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:09,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:09,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:09,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:09,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 16:56:09,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49fe2c8[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:09,295 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 16:56:09,295 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,296 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,296 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 16:56:09,296 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 16:56:09,296 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:09,296 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,296 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:09,298 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,298 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 16:56:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:09,298 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:09,298 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 16:56:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:09,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:09,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 16:56:09,308 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 16:56:09,308 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:09,308 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:09,308 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,308 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:09,310 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,310 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,310 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:09,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:09,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,311 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:09,312 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:09,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 16:56:09,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:09,316 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:09,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:09,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:09,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:09,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:09,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:09,316 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,317 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,317 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,417 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:09,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:09,419 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-13 16:56:09,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:56:09,421 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:56:09,421 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:56:09,421 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307969421 2023-02-13 16:56:09,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 16:56:09,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-13 16:56:09,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:09,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x71c8aa36 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:56:09,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4860a20d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:56:09,431 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:10,256 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036a8e745677a242f9aa389b71e1fadf43 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601036a8e745677a242f9aa389b71e1fadf43 2023-02-13 16:56:10,257 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:10,273 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:56:10,275 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52984, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:56:10,286 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:56:10,287 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41922, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:56:10,296 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/810a27fc926845f6ae3e03d87204bb61 first=Optional[r5] last=Optional[r6] 2023-02-13 16:56:10,296 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/810a27fc926845f6ae3e03d87204bb61}] 2023-02-13 16:56:10,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/810a27fc926845f6ae3e03d87204bb61 for inclusion in f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:56:10,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-13 16:56:10,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:56:10,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:10,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/810a27fc926845f6ae3e03d87204bb61 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__7uohmdpg9ltm9tgv6h18o1h7a7rk87uife8e93a41aq8f1lntfm9jm1c7inasiju/family1/810a27fc926845f6ae3e03d87204bb61 2023-02-13 16:56:10,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__7uohmdpg9ltm9tgv6h18o1h7a7rk87uife8e93a41aq8f1lntfm9jm1c7inasiju/family1/810a27fc926845f6ae3e03d87204bb61 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_ 2023-02-13 16:56:10,313 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__7uohmdpg9ltm9tgv6h18o1h7a7rk87uife8e93a41aq8f1lntfm9jm1c7inasiju/family1/810a27fc926845f6ae3e03d87204bb61 into f662746060867d63effc2a9ff57f8b73/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_ - updating store file list. 2023-02-13 16:56:10,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_ into f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:56:10,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__7uohmdpg9ltm9tgv6h18o1h7a7rk87uife8e93a41aq8f1lntfm9jm1c7inasiju/family1/810a27fc926845f6ae3e03d87204bb61 into f662746060867d63effc2a9ff57f8b73/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_) 2023-02-13 16:56:10,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__7uohmdpg9ltm9tgv6h18o1h7a7rk87uife8e93a41aq8f1lntfm9jm1c7inasiju/family1/810a27fc926845f6ae3e03d87204bb61 2023-02-13 16:56:10,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 16:56:10,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 16:56:10,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(1912): f662746060867d63effc2a9ff57f8b73/family1 is initiating minor compaction 2023-02-13 16:56:10,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4a6506eb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:10,320 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f662746060867d63effc2a9ff57f8b73/family1 in testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:10,320 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c2765a67910445cf893236d91dd63ce5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/f7902c3a9ef04649b5edac4d5f43a929, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9056403cdb90485c8505e6559a0c2ee8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/5c762176de874ca4b03d52568ef88558, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/61ac595e258846f0ba64205cdcf60007, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9d6f62539b3b41aaa4fdf01e1eb7b230, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/662147028b304acab281ea053671ad24, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/b24357a9fd604b6ca38c6db36bba8551, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/26aeda4f1ba0434a80d70d905453511c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp, totalSize=49.7 K 2023-02-13 16:56:10,320 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting c2765a67910445cf893236d91dd63ce5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 16:56:10,320 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting f7902c3a9ef04649b5edac4d5f43a929, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 9056403cdb90485c8505e6559a0c2ee8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 5c762176de874ca4b03d52568ef88558, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 61ac595e258846f0ba64205cdcf60007, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 9d6f62539b3b41aaa4fdf01e1eb7b230, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 662147028b304acab281ea053671ad24, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting b24357a9fd604b6ca38c6db36bba8551, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 26aeda4f1ba0434a80d70d905453511c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 16:56:10,321 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting bb70583eea5e455f96c9a02b23fede96_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 16:56:10,323 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:10,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:10,325 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601013d2be505248d452facd3df9c4413e333 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601013d2be505248d452facd3df9c4413e333 2023-02-13 16:56:10,326 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103250c8889d58a40b4ac0af411462f59bf to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103250c8889d58a40b4ac0af411462f59bf 2023-02-13 16:56:10,326 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-13 16:56:10,326 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-13 16:56:10,326 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:10,327 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:10,355 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f662746060867d63effc2a9ff57f8b73#family1#compaction#165 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:56:11,173 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/84ee2420069143fcb67586148625705b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/84ee2420069143fcb67586148625705b 2023-02-13 16:56:11,181 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123140fe6bd6305d41b382e20665d39dedea to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123140fe6bd6305d41b382e20665d39dedea 2023-02-13 16:56:11,182 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:11,182 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f662746060867d63effc2a9ff57f8b73/family1 of f662746060867d63effc2a9ff57f8b73 into 84ee2420069143fcb67586148625705b(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:11,182 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:11,182 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676307370320; duration=0sec 2023-02-13 16:56:11,182 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:11,214 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c7bc4e478191437ebf36ad253abccf9a first=Optional[r1] last=Optional[r4] 2023-02-13 16:56:11,215 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c7bc4e478191437ebf36ad253abccf9a}] 2023-02-13 16:56:11,217 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c7bc4e478191437ebf36ad253abccf9a for inclusion in f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:56:11,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-13 16:56:11,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:56:11,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:11,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c7bc4e478191437ebf36ad253abccf9a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gvqivalkjpmq8133b4jlgmco74gfbr3jok9ju31e2dams704ce8054ea65u4h2dk/family1/c7bc4e478191437ebf36ad253abccf9a 2023-02-13 16:56:11,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gvqivalkjpmq8133b4jlgmco74gfbr3jok9ju31e2dams704ce8054ea65u4h2dk/family1/c7bc4e478191437ebf36ad253abccf9a as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_ 2023-02-13 16:56:11,223 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gvqivalkjpmq8133b4jlgmco74gfbr3jok9ju31e2dams704ce8054ea65u4h2dk/family1/c7bc4e478191437ebf36ad253abccf9a into f662746060867d63effc2a9ff57f8b73/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_ - updating store file list. 2023-02-13 16:56:11,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_ into f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:56:11,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gvqivalkjpmq8133b4jlgmco74gfbr3jok9ju31e2dams704ce8054ea65u4h2dk/family1/c7bc4e478191437ebf36ad253abccf9a into f662746060867d63effc2a9ff57f8b73/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_) 2023-02-13 16:56:11,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gvqivalkjpmq8133b4jlgmco74gfbr3jok9ju31e2dams704ce8054ea65u4h2dk/family1/c7bc4e478191437ebf36ad253abccf9a 2023-02-13 16:56:11,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 16:56:11,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 16:56:11,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(1912): f662746060867d63effc2a9ff57f8b73/family1 is initiating minor compaction (all files) 2023-02-13 16:56:11,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@423aff60; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:11,228 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f662746060867d63effc2a9ff57f8b73/family1 in testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:11,228 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c94d0e8470094af0bcdaad42947c83fe, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/84ee2420069143fcb67586148625705b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp, totalSize=178.1 K 2023-02-13 16:56:11,229 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting c94d0e8470094af0bcdaad42947c83fe, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842816 2023-02-13 16:56:11,230 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting 84ee2420069143fcb67586148625705b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242816 2023-02-13 16:56:11,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:11,230 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] compactions.Compactor(207): Compacting cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242816 2023-02-13 16:56:11,231 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:11,234 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512015b070fc9da664020929030e1752e8fd4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512015b070fc9da664020929030e1752e8fd4 2023-02-13 16:56:11,235 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205470379edc13a4a9ab0e94650b723b423 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205470379edc13a4a9ab0e94650b723b423 2023-02-13 16:56:11,237 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228889e6067716247aeb3c85691ab5c64a4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228889e6067716247aeb3c85691ab5c64a4 2023-02-13 16:56:11,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c2765a67910445cf893236d91dd63ce5, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/f7902c3a9ef04649b5edac4d5f43a929, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9056403cdb90485c8505e6559a0c2ee8, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/5c762176de874ca4b03d52568ef88558, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/61ac595e258846f0ba64205cdcf60007, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9d6f62539b3b41aaa4fdf01e1eb7b230, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/662147028b304acab281ea053671ad24, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/b24357a9fd604b6ca38c6db36bba8551, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/26aeda4f1ba0434a80d70d905453511c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_] to archive 2023-02-13 16:56:11,240 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231eef38173a1cc4c4da990de5ff851a689 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231eef38173a1cc4c4da990de5ff851a689 2023-02-13 16:56:11,240 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-13 16:56:11,240 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-13 16:56:11,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:11,241 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:11,241 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:11,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c2765a67910445cf893236d91dd63ce5 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c2765a67910445cf893236d91dd63ce5 2023-02-13 16:56:11,245 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/f7902c3a9ef04649b5edac4d5f43a929 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/f7902c3a9ef04649b5edac4d5f43a929 2023-02-13 16:56:11,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9056403cdb90485c8505e6559a0c2ee8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9056403cdb90485c8505e6559a0c2ee8 2023-02-13 16:56:11,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/5c762176de874ca4b03d52568ef88558 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/5c762176de874ca4b03d52568ef88558 2023-02-13 16:56:11,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/61ac595e258846f0ba64205cdcf60007 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/61ac595e258846f0ba64205cdcf60007 2023-02-13 16:56:11,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9d6f62539b3b41aaa4fdf01e1eb7b230 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/9d6f62539b3b41aaa4fdf01e1eb7b230 2023-02-13 16:56:11,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/662147028b304acab281ea053671ad24 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/662147028b304acab281ea053671ad24 2023-02-13 16:56:11,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/b24357a9fd604b6ca38c6db36bba8551 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/b24357a9fd604b6ca38c6db36bba8551 2023-02-13 16:56:11,260 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/26aeda4f1ba0434a80d70d905453511c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/26aeda4f1ba0434a80d70d905453511c 2023-02-13 16:56:11,263 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/bb70583eea5e455f96c9a02b23fede96_SeqId_44_ 2023-02-13 16:56:11,276 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f662746060867d63effc2a9ff57f8b73#family1#compaction#166 average throughput is 9.73 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 16:56:11,318 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113089141ec3a9ec49d8833902f5de2cca2f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113089141ec3a9ec49d8833902f5de2cca2f 2023-02-13 16:56:11,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:11,355 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 first=Optional[r00] last=Optional[r8] 2023-02-13 16:56:11,355 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2}] 2023-02-13 16:56:11,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 for inclusion in f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:56:11,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-13 16:56:11,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 16:56:11,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:11,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gtbd4e66j160cesnt9u4uc3jdefj2oslvjrrggikff5okn5f6geuntf9frmmaqam/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 2023-02-13 16:56:11,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gtbd4e66j160cesnt9u4uc3jdefj2oslvjrrggikff5okn5f6geuntf9frmmaqam/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/750898f327634344947f97c91548ed29_SeqId_49_ 2023-02-13 16:56:11,363 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gtbd4e66j160cesnt9u4uc3jdefj2oslvjrrggikff5okn5f6geuntf9frmmaqam/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 into f662746060867d63effc2a9ff57f8b73/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/750898f327634344947f97c91548ed29_SeqId_49_ - updating store file list. 2023-02-13 16:56:11,366 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/750898f327634344947f97c91548ed29_SeqId_49_ into f662746060867d63effc2a9ff57f8b73/family1 2023-02-13 16:56:11,366 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gtbd4e66j160cesnt9u4uc3jdefj2oslvjrrggikff5okn5f6geuntf9frmmaqam/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 into f662746060867d63effc2a9ff57f8b73/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/750898f327634344947f97c91548ed29_SeqId_49_) 2023-02-13 16:56:11,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gtbd4e66j160cesnt9u4uc3jdefj2oslvjrrggikff5okn5f6geuntf9frmmaqam/family1/ae51b864fe8b4c1bbe04685f4f3cc7c2 2023-02-13 16:56:11,369 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:11,369 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:11,370 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120ed36f2c1b8744149bffb936e4227234c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120ed36f2c1b8744149bffb936e4227234c 2023-02-13 16:56:11,371 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112859306fd57a5d48c29e39a84b915b9039 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112859306fd57a5d48c29e39a84b915b9039 2023-02-13 16:56:11,372 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130515cfdc2b1994626803105817c98e8df to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130515cfdc2b1994626803105817c98e8df 2023-02-13 16:56:11,372 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-13 16:56:11,373 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 16:56:11,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:11,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:11,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 16:56:11,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 16:56:11,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:56:11,376 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 16:56:11,377 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9570 sec 2023-02-13 16:56:11,617 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255 (was 246) - Thread LEAK? -, OpenFileDescriptor=644 (was 630) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=99 (was 117), ProcessCount=168 (was 168), AvailableMemoryMB=1026 (was 1103) 2023-02-13 16:56:11,630 INFO [Listener at localhost/36833] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=255, OpenFileDescriptor=644, MaxFileDescriptor=60000, SystemLoadAverage=99, ProcessCount=168, AvailableMemoryMB=1026 2023-02-13 16:56:11,650 DEBUG [Listener at localhost/36833] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-13 16:56:11,703 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 16:56:11,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-13 16:56:11,706 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 16:56:11,706 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-13 16:56:11,706 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 16:56:11,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 16:56:11,708 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:11,708 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:11,708 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:11,709 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187 empty. 2023-02-13 16:56:11,709 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601 empty. 2023-02-13 16:56:11,709 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909 empty. 2023-02-13 16:56:11,709 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:11,709 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:11,709 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:11,709 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-13 16:56:11,722 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/.tmp/family1/7b312c4af03d4badbadb4ab535b2282f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/7b312c4af03d4badbadb4ab535b2282f 2023-02-13 16:56:11,725 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f662746060867d63effc2a9ff57f8b73/family1 of f662746060867d63effc2a9ff57f8b73 into 7b312c4af03d4badbadb4ab535b2282f(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:11,725 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f662746060867d63effc2a9ff57f8b73: 2023-02-13 16:56:11,725 INFO [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676307371228; duration=0sec 2023-02-13 16:56:11,725 DEBUG [RS:0;jenkins-hbase4:33907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 16:56:11,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c94d0e8470094af0bcdaad42947c83fe, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/84ee2420069143fcb67586148625705b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_] to archive 2023-02-13 16:56:11,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:11,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c94d0e8470094af0bcdaad42947c83fe to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/c94d0e8470094af0bcdaad42947c83fe 2023-02-13 16:56:11,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/84ee2420069143fcb67586148625705b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/84ee2420069143fcb67586148625705b 2023-02-13 16:56:11,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_ to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/family1/cbd4ab3a674a4b1aae5d1995bad74ba9_SeqId_47_ 2023-02-13 16:56:11,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 16:56:12,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 16:56:12,116 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-13 16:56:12,117 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => b21a40fdb4ff985de26173896bbc0187, NAME => 'testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:56:12,121 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => c346712ffdda196120c65a73d77e9601, NAME => 'testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:56:12,121 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => ce2e1b6d8e70b199cea2ad6e7d3e5909, NAME => 'testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/.tmp 2023-02-13 16:56:12,136 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,136 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing b21a40fdb4ff985de26173896bbc0187, disabling compactions & flushes 2023-02-13 16:56:12,136 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,136 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,136 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. after waiting 0 ms 2023-02-13 16:56:12,136 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,137 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,137 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for b21a40fdb4ff985de26173896bbc0187: Waiting for close lock at 1676307372136Disabling compacts and flushes for region at 1676307372136Disabling writes for close at 1676307372136Writing region close event to WAL at 1676307372137 (+1 ms)Closed at 1676307372137 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing ce2e1b6d8e70b199cea2ad6e7d3e5909, disabling compactions & flushes 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,143 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing c346712ffdda196120c65a73d77e9601, disabling compactions & flushes 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,143 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. after waiting 0 ms 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. after waiting 0 ms 2023-02-13 16:56:12,143 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: Waiting for close lock at 1676307372143Disabling compacts and flushes for region at 1676307372143Disabling writes for close at 1676307372143Writing region close event to WAL at 1676307372143Closed at 1676307372143 2023-02-13 16:56:12,143 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,143 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for c346712ffdda196120c65a73d77e9601: Waiting for close lock at 1676307372143Disabling compacts and flushes for region at 1676307372143Disabling writes for close at 1676307372143Writing region close event to WAL at 1676307372143Closed at 1676307372143 2023-02-13 16:56:12,145 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 16:56:12,146 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307372146"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307372146"}]},"ts":"1676307372146"} 2023-02-13 16:56:12,146 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307372146"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307372146"}]},"ts":"1676307372146"} 2023-02-13 16:56:12,147 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307372146"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676307372146"}]},"ts":"1676307372146"} 2023-02-13 16:56:12,148 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 16:56:12,149 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 16:56:12,149 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307372149"}]},"ts":"1676307372149"} 2023-02-13 16:56:12,150 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-13 16:56:12,153 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b21a40fdb4ff985de26173896bbc0187, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce2e1b6d8e70b199cea2ad6e7d3e5909, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=c346712ffdda196120c65a73d77e9601, ASSIGN}] 2023-02-13 16:56:12,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce2e1b6d8e70b199cea2ad6e7d3e5909, ASSIGN 2023-02-13 16:56:12,155 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b21a40fdb4ff985de26173896bbc0187, ASSIGN 2023-02-13 16:56:12,155 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=c346712ffdda196120c65a73d77e9601, ASSIGN 2023-02-13 16:56:12,155 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b21a40fdb4ff985de26173896bbc0187, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:56:12,155 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce2e1b6d8e70b199cea2ad6e7d3e5909, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:56:12,155 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=c346712ffdda196120c65a73d77e9601, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33907,1676307219708; forceNewPlan=false, retain=false 2023-02-13 16:56:12,307 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=c346712ffdda196120c65a73d77e9601, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,307 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ce2e1b6d8e70b199cea2ad6e7d3e5909, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,307 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307372307"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307372307"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307372307"}]},"ts":"1676307372307"} 2023-02-13 16:56:12,307 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=b21a40fdb4ff985de26173896bbc0187, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,307 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307372307"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307372307"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307372307"}]},"ts":"1676307372307"} 2023-02-13 16:56:12,307 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307372307"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676307372307"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676307372307"}]},"ts":"1676307372307"} 2023-02-13 16:56:12,309 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure c346712ffdda196120c65a73d77e9601, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:56:12,309 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure ce2e1b6d8e70b199cea2ad6e7d3e5909, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:56:12,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 16:56:12,310 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=117, state=RUNNABLE; OpenRegionProcedure b21a40fdb4ff985de26173896bbc0187, server=jenkins-hbase4.apache.org,33907,1676307219708}] 2023-02-13 16:56:12,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c346712ffdda196120c65a73d77e9601, NAME => 'testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 16:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,484 INFO [StoreOpener-c346712ffdda196120c65a73d77e9601-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,486 DEBUG [StoreOpener-c346712ffdda196120c65a73d77e9601-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1 2023-02-13 16:56:12,486 DEBUG [StoreOpener-c346712ffdda196120c65a73d77e9601-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1 2023-02-13 16:56:12,486 INFO [StoreOpener-c346712ffdda196120c65a73d77e9601-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c346712ffdda196120c65a73d77e9601 columnFamilyName family1 2023-02-13 16:56:12,487 INFO [StoreOpener-c346712ffdda196120c65a73d77e9601-1] regionserver.HStore(310): Store=c346712ffdda196120c65a73d77e9601/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,488 INFO [StoreOpener-c346712ffdda196120c65a73d77e9601-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,488 DEBUG [StoreOpener-c346712ffdda196120c65a73d77e9601-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2 2023-02-13 16:56:12,489 DEBUG [StoreOpener-c346712ffdda196120c65a73d77e9601-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2 2023-02-13 16:56:12,489 INFO [StoreOpener-c346712ffdda196120c65a73d77e9601-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c346712ffdda196120c65a73d77e9601 columnFamilyName family2 2023-02-13 16:56:12,490 INFO [StoreOpener-c346712ffdda196120c65a73d77e9601-1] regionserver.HStore(310): Store=c346712ffdda196120c65a73d77e9601/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:12,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:12,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:56:12,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c346712ffdda196120c65a73d77e9601; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73669814, jitterRate=0.09776577353477478}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:12,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c346712ffdda196120c65a73d77e9601: Running coprocessor pre-open hook at 1676307372463Writing region info on filesystem at 1676307372463Initializing all the Stores at 1676307372464 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307372464Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307372464Cleaning up temporary data from old regions at 1676307372491 (+27 ms)Cleaning up detritus from prior splits at 1676307372491Running coprocessor post-open hooks at 1676307372494 (+3 ms)Region opened successfully at 1676307372494 2023-02-13 16:56:12,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601., pid=120, masterSystemTime=1676307372460 2023-02-13 16:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ce2e1b6d8e70b199cea2ad6e7d3e5909, NAME => 'testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 16:56:12,497 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=c346712ffdda196120c65a73d77e9601, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,497 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307372497"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307372497"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307372497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307372497"}]},"ts":"1676307372497"} 2023-02-13 16:56:12,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,499 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-13 16:56:12,499 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure c346712ffdda196120c65a73d77e9601, server=jenkins-hbase4.apache.org,33907,1676307219708 in 190 msec 2023-02-13 16:56:12,500 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=c346712ffdda196120c65a73d77e9601, ASSIGN in 346 msec 2023-02-13 16:56:12,517 INFO [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,518 DEBUG [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 2023-02-13 16:56:12,518 DEBUG [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 2023-02-13 16:56:12,519 INFO [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ce2e1b6d8e70b199cea2ad6e7d3e5909 columnFamilyName family1 2023-02-13 16:56:12,519 INFO [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] regionserver.HStore(310): Store=ce2e1b6d8e70b199cea2ad6e7d3e5909/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,520 INFO [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,520 DEBUG [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2 2023-02-13 16:56:12,521 DEBUG [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2 2023-02-13 16:56:12,521 INFO [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ce2e1b6d8e70b199cea2ad6e7d3e5909 columnFamilyName family2 2023-02-13 16:56:12,521 INFO [StoreOpener-ce2e1b6d8e70b199cea2ad6e7d3e5909-1] regionserver.HStore(310): Store=ce2e1b6d8e70b199cea2ad6e7d3e5909/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:12,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:12,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:56:12,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ce2e1b6d8e70b199cea2ad6e7d3e5909; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75489603, jitterRate=0.12488274276256561}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:12,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: Running coprocessor pre-open hook at 1676307372497Writing region info on filesystem at 1676307372497Initializing all the Stores at 1676307372497Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307372498 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307372498Cleaning up temporary data from old regions at 1676307372523 (+25 ms)Cleaning up detritus from prior splits at 1676307372523Running coprocessor post-open hooks at 1676307372526 (+3 ms)Region opened successfully at 1676307372526 2023-02-13 16:56:12,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909., pid=121, masterSystemTime=1676307372460 2023-02-13 16:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b21a40fdb4ff985de26173896bbc0187, NAME => 'testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 16:56:12,528 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ce2e1b6d8e70b199cea2ad6e7d3e5909, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 16:56:12,528 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676307372528"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307372528"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307372528"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307372528"}]},"ts":"1676307372528"} 2023-02-13 16:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,530 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-02-13 16:56:12,530 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure ce2e1b6d8e70b199cea2ad6e7d3e5909, server=jenkins-hbase4.apache.org,33907,1676307219708 in 220 msec 2023-02-13 16:56:12,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce2e1b6d8e70b199cea2ad6e7d3e5909, ASSIGN in 377 msec 2023-02-13 16:56:12,549 INFO [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,550 DEBUG [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1 2023-02-13 16:56:12,551 DEBUG [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1 2023-02-13 16:56:12,551 INFO [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b21a40fdb4ff985de26173896bbc0187 columnFamilyName family1 2023-02-13 16:56:12,552 INFO [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] regionserver.HStore(310): Store=b21a40fdb4ff985de26173896bbc0187/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,552 INFO [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,553 DEBUG [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2 2023-02-13 16:56:12,553 DEBUG [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2 2023-02-13 16:56:12,553 INFO [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b21a40fdb4ff985de26173896bbc0187 columnFamilyName family2 2023-02-13 16:56:12,554 INFO [StoreOpener-b21a40fdb4ff985de26173896bbc0187-1] regionserver.HStore(310): Store=b21a40fdb4ff985de26173896bbc0187/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 16:56:12,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 16:56:12,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:12,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 16:56:12,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b21a40fdb4ff985de26173896bbc0187; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69854832, jitterRate=0.04091811180114746}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 16:56:12,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b21a40fdb4ff985de26173896bbc0187: Running coprocessor pre-open hook at 1676307372528Writing region info on filesystem at 1676307372528Initializing all the Stores at 1676307372529 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307372529Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676307372529Cleaning up temporary data from old regions at 1676307372555 (+26 ms)Cleaning up detritus from prior splits at 1676307372556 (+1 ms)Running coprocessor post-open hooks at 1676307372559 (+3 ms)Region opened successfully at 1676307372559 2023-02-13 16:56:12,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187., pid=122, masterSystemTime=1676307372460 2023-02-13 16:56:12,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,561 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=b21a40fdb4ff985de26173896bbc0187, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,561 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676307372561"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676307372561"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676307372561"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676307372561"}]},"ts":"1676307372561"} 2023-02-13 16:56:12,564 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=117 2023-02-13 16:56:12,564 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=117, state=SUCCESS; OpenRegionProcedure b21a40fdb4ff985de26173896bbc0187, server=jenkins-hbase4.apache.org,33907,1676307219708 in 253 msec 2023-02-13 16:56:12,566 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-13 16:56:12,566 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b21a40fdb4ff985de26173896bbc0187, ASSIGN in 411 msec 2023-02-13 16:56:12,566 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 16:56:12,566 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676307372566"}]},"ts":"1676307372566"} 2023-02-13 16:56:12,567 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-13 16:56:12,570 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 16:56:12,571 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 867 msec 2023-02-13 16:56:12,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 16:56:12,811 INFO [Listener at localhost/36833] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-13 16:56:12,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:12,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:12,818 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:12,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:12,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 16:56:12,821 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 16:56:12,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:12,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 16:56:12,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 16:56:12,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:12,821 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:12,822 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:12,822 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:12,823 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:12,823 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:12,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:12,823 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 16:56:12,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 16:56:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 16:56:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 16:56:12,824 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:12,824 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. started... 2023-02-13 16:56:12,824 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:12,824 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. started... 2023-02-13 16:56:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:56:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:12,824 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:12,825 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing b21a40fdb4ff985de26173896bbc0187 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:56:12,825 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ce2e1b6d8e70b199cea2ad6e7d3e5909 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:56:12,826 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. started... 2023-02-13 16:56:12,826 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c346712ffdda196120c65a73d77e9601 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 16:56:12,912 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021321ee2d8f4e4f4df7bd8d1631c21e1d05 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021321ee2d8f4e4f4df7bd8d1631c21e1d05 2023-02-13 16:56:12,914 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302130b1d5c79956242e195ce977ccc675f57 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302130b1d5c79956242e195ce977ccc675f57 2023-02-13 16:56:13,287 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133d8d2a342fe443fc88e2e43ac9b596a7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302133d8d2a342fe443fc88e2e43ac9b596a7 2023-02-13 16:56:13,332 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/bcf8b0e8913f4a31afff7dbc65983844 2023-02-13 16:56:13,333 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/7b021715bf4447a2abad724cde654e9b 2023-02-13 16:56:13,714 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/be0919d2895b4a38b9cbb7687c1a4305 2023-02-13 16:56:13,746 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213003cf00a297541b5bdf5e13271620857 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213003cf00a297541b5bdf5e13271620857 2023-02-13 16:56:13,766 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021366372bfcc3944c118c4fe4d59937bd35 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023021366372bfcc3944c118c4fe4d59937bd35 2023-02-13 16:56:13,767 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302134fa4f3a5523848c2b81e370369596b9d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302134fa4f3a5523848c2b81e370369596b9d 2023-02-13 16:56:14,150 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/a85e7522e9764045bc9c92eebe70c420 2023-02-13 16:56:14,154 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/be0919d2895b4a38b9cbb7687c1a4305 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/be0919d2895b4a38b9cbb7687c1a4305 2023-02-13 16:56:14,171 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/bceeb6aef4cf496183d5f1f997f3f015 2023-02-13 16:56:14,171 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/3a56f97577df4eaa99d20c418c49d522 2023-02-13 16:56:14,175 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/bcf8b0e8913f4a31afff7dbc65983844 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/bcf8b0e8913f4a31afff7dbc65983844 2023-02-13 16:56:14,175 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/7b021715bf4447a2abad724cde654e9b as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/7b021715bf4447a2abad724cde654e9b 2023-02-13 16:56:14,178 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/be0919d2895b4a38b9cbb7687c1a4305, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 16:56:14,179 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/a85e7522e9764045bc9c92eebe70c420 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/a85e7522e9764045bc9c92eebe70c420 2023-02-13 16:56:14,183 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/a85e7522e9764045bc9c92eebe70c420, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:56:14,183 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b21a40fdb4ff985de26173896bbc0187 in 1358ms, sequenceid=5, compaction requested=false 2023-02-13 16:56:14,184 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-13 16:56:14,184 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:14,184 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:14,184 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:56:14,201 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/7b021715bf4447a2abad724cde654e9b, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 16:56:14,201 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/3a56f97577df4eaa99d20c418c49d522 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/3a56f97577df4eaa99d20c418c49d522 2023-02-13 16:56:14,202 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/bcf8b0e8913f4a31afff7dbc65983844, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 16:56:14,202 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/bceeb6aef4cf496183d5f1f997f3f015 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/bceeb6aef4cf496183d5f1f997f3f015 2023-02-13 16:56:14,205 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/3a56f97577df4eaa99d20c418c49d522, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:56:14,205 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/bceeb6aef4cf496183d5f1f997f3f015, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 16:56:14,206 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c346712ffdda196120c65a73d77e9601 in 1380ms, sequenceid=5, compaction requested=false 2023-02-13 16:56:14,206 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ce2e1b6d8e70b199cea2ad6e7d3e5909 in 1381ms, sequenceid=5, compaction requested=false 2023-02-13 16:56:14,206 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:14,206 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:14,206 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:14,207 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:14,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:56:14,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:56:14,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:56:14,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 16:56:14,207 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 16:56:14,217 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,217 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:14,217 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,217 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:14,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:14,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:14,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:14,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 16:56:14,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@911badd[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:14,219 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 16:56:14,219 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,220 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,220 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 16:56:14,220 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 16:56:14,220 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:14,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,220 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:14,221 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 16:56:14,221 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:14,221 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,221 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 16:56:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:14,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 16:56:14,223 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 16:56:14,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:14,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:14,223 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 16:56:14,223 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:14,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:14,225 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,226 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:14,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:14,226 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:14,228 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,228 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:14,228 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:14,228 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:14,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:14,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 16:56:14,229 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:14,228 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:14,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:14,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 16:56:14,228 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:14,229 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,229 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:14,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:14,229 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:14,229 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:14,229 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:14,229 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,329 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:14,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:14,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:14,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:14,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:14,333 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:14,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:14,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 16:56:14,335 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 16:56:14,335 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:14,335 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 16:56:14,335 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 16:56:14,336 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,336 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:14,337 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,337 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:14,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:14,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:14,337 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:14,337 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:14,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:14,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:14,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:14,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:14,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 16:56:14,338 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 16:56:14,338 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 16:56:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 16:56:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 16:56:14,339 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:14,339 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. started... 2023-02-13 16:56:14,339 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ce2e1b6d8e70b199cea2ad6e7d3e5909 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:56:14,341 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:14,341 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:56:14,342 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:14,342 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. started... 2023-02-13 16:56:14,342 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:14,342 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c346712ffdda196120c65a73d77e9601 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:56:14,341 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. started... 2023-02-13 16:56:14,342 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing b21a40fdb4ff985de26173896bbc0187 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 16:56:14,784 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302138301e4b5242c4755830b055338eb8813 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302138301e4b5242c4755830b055338eb8813 2023-02-13 16:56:14,786 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302133061081a97cc4283aa60c9436671c4fd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302133061081a97cc4283aa60c9436671c4fd 2023-02-13 16:56:14,788 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213742b04eb556a4312b2def7201376e288 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213742b04eb556a4312b2def7201376e288 2023-02-13 16:56:14,804 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/b474ede3450e47d4a0a704f00cafc908 2023-02-13 16:56:15,202 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/6240ec83ac944b07bfe3d2c2a78059b7 2023-02-13 16:56:15,203 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/007425ccdeeb4f6d8b0509b0531e6b76 2023-02-13 16:56:15,238 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213de8e5a9628af40b2b6ab1aee2312af3c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213de8e5a9628af40b2b6ab1aee2312af3c 2023-02-13 16:56:15,262 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:56:15,263 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71c8aa36 to 127.0.0.1:64619 2023-02-13 16:56:15,263 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:15,263 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:56:15,263 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a0f9b20 to 127.0.0.1:64619 2023-02-13 16:56:15,263 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:15,268 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/fdab1b278e224d7194476a74a6cb9b13 2023-02-13 16:56:15,273 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021309dc2e9711bb44a6899a6710b849a2a4 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023021309dc2e9711bb44a6899a6710b849a2a4 2023-02-13 16:56:15,273 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/b474ede3450e47d4a0a704f00cafc908 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/b474ede3450e47d4a0a704f00cafc908 2023-02-13 16:56:15,275 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213f770e4759bb544c7935d40c7ca783a44 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213f770e4759bb544c7935d40c7ca783a44 2023-02-13 16:56:15,299 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/b474ede3450e47d4a0a704f00cafc908, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 16:56:15,300 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/fdab1b278e224d7194476a74a6cb9b13 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/fdab1b278e224d7194476a74a6cb9b13 2023-02-13 16:56:15,304 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/fdab1b278e224d7194476a74a6cb9b13, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:56:15,304 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ce2e1b6d8e70b199cea2ad6e7d3e5909 in 965ms, sequenceid=9, compaction requested=false 2023-02-13 16:56:15,304 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:15,304 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:56:15,686 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/2873f41693f94fbb891941141532cddd 2023-02-13 16:56:15,691 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/007425ccdeeb4f6d8b0509b0531e6b76 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/007425ccdeeb4f6d8b0509b0531e6b76 2023-02-13 16:56:15,691 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/af4d3f0b04f24e4c94079121bf352644 2023-02-13 16:56:15,695 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/6240ec83ac944b07bfe3d2c2a78059b7 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/6240ec83ac944b07bfe3d2c2a78059b7 2023-02-13 16:56:15,715 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/007425ccdeeb4f6d8b0509b0531e6b76, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 16:56:15,716 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/2873f41693f94fbb891941141532cddd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2873f41693f94fbb891941141532cddd 2023-02-13 16:56:15,719 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2873f41693f94fbb891941141532cddd, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:56:15,719 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/6240ec83ac944b07bfe3d2c2a78059b7, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 16:56:15,719 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b21a40fdb4ff985de26173896bbc0187 in 1377ms, sequenceid=9, compaction requested=false 2023-02-13 16:56:15,719 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/af4d3f0b04f24e4c94079121bf352644 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/af4d3f0b04f24e4c94079121bf352644 2023-02-13 16:56:15,719 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:15,719 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:15,720 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:56:15,723 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/af4d3f0b04f24e4c94079121bf352644, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 16:56:15,723 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c346712ffdda196120c65a73d77e9601 in 1381ms, sequenceid=9, compaction requested=false 2023-02-13 16:56:15,723 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:15,723 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:15,723 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:56:15,723 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:56:15,723 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,724 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 16:56:15,724 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 16:56:15,725 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,725 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:15,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:15,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:15,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:15,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:15,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:15,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 16:56:15,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75bb6d95[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:15,727 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 16:56:15,727 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,729 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,729 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 16:56:15,729 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 16:56:15,729 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:15,729 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,730 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:15,731 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,731 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 16:56:15,731 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:15,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,731 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 16:56:15,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:15,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:15,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:15,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:15,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:15,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:15,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:15,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 16:56:15,733 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 16:56:15,733 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:15,733 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:15,733 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 16:56:15,733 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:15,735 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,735 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,735 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:15,735 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:15,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:15,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:15,736 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:15,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,738 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:15,738 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:15,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 16:56:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:15,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:15,738 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:15,738 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:15,738 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:15,739 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:15,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 16:56:15,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:15,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:15,839 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:15,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:15,841 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:15,842 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:15,842 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:15,844 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:15,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:15,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 16:56:15,846 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 16:56:15,846 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:15,846 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 16:56:15,846 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 16:56:15,847 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,847 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:15,850 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:15,850 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:15,850 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:15,850 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:15,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:15,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 16:56:15,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 16:56:15,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 16:56:15,851 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 16:56:15,851 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:15,851 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 16:56:15,852 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:15,852 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. started... 2023-02-13 16:56:15,852 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:15,852 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. started... 2023-02-13 16:56:15,852 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ce2e1b6d8e70b199cea2ad6e7d3e5909 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:56:15,852 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:56:15,852 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:15,852 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:15,852 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing b21a40fdb4ff985de26173896bbc0187 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:56:15,852 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. started... 2023-02-13 16:56:15,853 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c346712ffdda196120c65a73d77e9601 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 16:56:15,934 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302130b855384f3ee449b8cb3fd3ff8f2487a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302130b855384f3ee449b8cb3fd3ff8f2487a 2023-02-13 16:56:15,945 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213768895e483314916bfce59ef9443c351 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213768895e483314916bfce59ef9443c351 2023-02-13 16:56:15,969 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/509167eb4636427ebca6fc21d740df81 2023-02-13 16:56:15,971 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/93376628c3bf4a42ad1676f0ba0bc7b6 2023-02-13 16:56:16,013 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021361b4c64776984df993cc300baacf829c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023021361b4c64776984df993cc300baacf829c 2023-02-13 16:56:16,019 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/310354f2b3704e0fa834f637c227efdc 2023-02-13 16:56:16,024 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/509167eb4636427ebca6fc21d740df81 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/509167eb4636427ebca6fc21d740df81 2023-02-13 16:56:16,057 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/509167eb4636427ebca6fc21d740df81, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 16:56:16,058 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/310354f2b3704e0fa834f637c227efdc as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/310354f2b3704e0fa834f637c227efdc 2023-02-13 16:56:16,062 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/310354f2b3704e0fa834f637c227efdc, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:56:16,063 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ce2e1b6d8e70b199cea2ad6e7d3e5909 in 211ms, sequenceid=13, compaction requested=true 2023-02-13 16:56:16,063 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:16,063 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:16,063 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:56:16,335 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302135c07db5363f1414eb85f4e7f8dece714 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135c07db5363f1414eb85f4e7f8dece714 2023-02-13 16:56:16,413 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133913d31b74e14c0da9b5193c927763bc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302133913d31b74e14c0da9b5193c927763bc 2023-02-13 16:56:16,421 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/c0d377521ae340a195559b824c24062c 2023-02-13 16:56:16,426 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/93376628c3bf4a42ad1676f0ba0bc7b6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/93376628c3bf4a42ad1676f0ba0bc7b6 2023-02-13 16:56:16,462 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/93376628c3bf4a42ad1676f0ba0bc7b6, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 16:56:16,463 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/c0d377521ae340a195559b824c24062c as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/c0d377521ae340a195559b824c24062c 2023-02-13 16:56:16,467 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/c0d377521ae340a195559b824c24062c, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:56:16,468 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b21a40fdb4ff985de26173896bbc0187 in 616ms, sequenceid=13, compaction requested=true 2023-02-13 16:56:16,468 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:16,468 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:16,468 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:56:16,763 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/d94500fd8c114862bae5e19d286829a3 2023-02-13 16:56:17,210 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302133023b351107e4f40b38ac6cb404d705e to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302133023b351107e4f40b38ac6cb404d705e 2023-02-13 16:56:17,615 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/4f9ebc1486e841df996c78202413cacc 2023-02-13 16:56:17,619 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/d94500fd8c114862bae5e19d286829a3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/d94500fd8c114862bae5e19d286829a3 2023-02-13 16:56:17,641 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/d94500fd8c114862bae5e19d286829a3, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 16:56:17,642 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/4f9ebc1486e841df996c78202413cacc as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/4f9ebc1486e841df996c78202413cacc 2023-02-13 16:56:17,645 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/4f9ebc1486e841df996c78202413cacc, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 16:56:17,646 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c346712ffdda196120c65a73d77e9601 in 1793ms, sequenceid=13, compaction requested=true 2023-02-13 16:56:17,646 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:17,646 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:17,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:56:17,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:56:17,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 16:56:17,646 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 16:56:17,648 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,648 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:17,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:17,648 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,649 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:17,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 16:56:17,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aaf3d14[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:17,650 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 16:56:17,650 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,651 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,651 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 16:56:17,651 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 16:56:17,651 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:17,651 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,651 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 16:56:17,654 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 16:56:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:17,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 16:56:17,656 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 16:56:17,656 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:17,656 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:17,656 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 16:56:17,656 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:17,658 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,658 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:17,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,658 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:17,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:17,658 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,659 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:17,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:17,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,661 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:17,661 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:17,661 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 16:56:17,662 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:17,661 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:17,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:17,661 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:17,662 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:17,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:17,762 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:17,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:17,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:17,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:17,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 16:56:17,767 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:17,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:17,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 16:56:17,769 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 16:56:17,769 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:17,769 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 16:56:17,769 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 16:56:17,770 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,770 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:17,782 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:17,782 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:17,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:17,782 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:17,782 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:17,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:17,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:17,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:17,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:17,783 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 16:56:17,784 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 16:56:17,784 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 16:56:17,784 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 16:56:17,784 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:17,784 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 16:56:17,784 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:17,784 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. started... 2023-02-13 16:56:17,784 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:17,784 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:56:17,784 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:17,784 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:17,784 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ce2e1b6d8e70b199cea2ad6e7d3e5909 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:56:17,784 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. started... 2023-02-13 16:56:17,784 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. started... 2023-02-13 16:56:17,785 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing b21a40fdb4ff985de26173896bbc0187 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:56:17,785 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c346712ffdda196120c65a73d77e9601 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 16:56:17,853 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213d18d1823b96748b3b2e1e39465d6ee91 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d18d1823b96748b3b2e1e39465d6ee91 2023-02-13 16:56:18,024 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 16:56:18,252 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302134250f761cf0142d38739e67db1fbbc4d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134250f761cf0142d38739e67db1fbbc4d 2023-02-13 16:56:18,257 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021328d4edac64b84a2a80496f80ae8a19ee to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021328d4edac64b84a2a80496f80ae8a19ee 2023-02-13 16:56:18,271 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/b0d30f156a3644a89b3659be40a01ce0 2023-02-13 16:56:18,271 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/1cf93ed1a45c45a9a43c9e5c7cce316d 2023-02-13 16:56:18,288 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/2f040cc53d324d4185580771ca2282fc 2023-02-13 16:56:18,315 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302131a92a8c39c2b41889d9211f37a8d14ab to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302131a92a8c39c2b41889d9211f37a8d14ab 2023-02-13 16:56:18,328 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213b15dc770eee24808874e956109e169f6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213b15dc770eee24808874e956109e169f6 2023-02-13 16:56:18,348 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/36831349709943ff8b8c1bc0c35a03b9 2023-02-13 16:56:18,348 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/3a738f6fda5e46b99d15c233047fff9f 2023-02-13 16:56:18,352 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/b0d30f156a3644a89b3659be40a01ce0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/b0d30f156a3644a89b3659be40a01ce0 2023-02-13 16:56:18,354 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/1cf93ed1a45c45a9a43c9e5c7cce316d as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/1cf93ed1a45c45a9a43c9e5c7cce316d 2023-02-13 16:56:18,395 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/b0d30f156a3644a89b3659be40a01ce0, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 16:56:18,397 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/36831349709943ff8b8c1bc0c35a03b9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/36831349709943ff8b8c1bc0c35a03b9 2023-02-13 16:56:18,397 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/1cf93ed1a45c45a9a43c9e5c7cce316d, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 16:56:18,398 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/3a738f6fda5e46b99d15c233047fff9f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/3a738f6fda5e46b99d15c233047fff9f 2023-02-13 16:56:18,402 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/36831349709943ff8b8c1bc0c35a03b9, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:56:18,403 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b21a40fdb4ff985de26173896bbc0187 in 618ms, sequenceid=17, compaction requested=true 2023-02-13 16:56:18,403 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:18,404 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:18,404 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:56:18,404 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/3a738f6fda5e46b99d15c233047fff9f, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:56:18,405 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ce2e1b6d8e70b199cea2ad6e7d3e5909 in 621ms, sequenceid=17, compaction requested=true 2023-02-13 16:56:18,405 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:18,405 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:18,405 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:56:18,754 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021391e81adaee7f4ec28d72c2bbe77288d7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023021391e81adaee7f4ec28d72c2bbe77288d7 2023-02-13 16:56:19,159 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/6d1a4c3553bb4e00b0907009b83cb545 2023-02-13 16:56:19,163 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/2f040cc53d324d4185580771ca2282fc as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/2f040cc53d324d4185580771ca2282fc 2023-02-13 16:56:19,186 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/2f040cc53d324d4185580771ca2282fc, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 16:56:19,187 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/6d1a4c3553bb4e00b0907009b83cb545 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/6d1a4c3553bb4e00b0907009b83cb545 2023-02-13 16:56:19,191 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/6d1a4c3553bb4e00b0907009b83cb545, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 16:56:19,192 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c346712ffdda196120c65a73d77e9601 in 1407ms, sequenceid=17, compaction requested=true 2023-02-13 16:56:19,192 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:19,192 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:19,192 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:56:19,192 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:56:19,192 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,192 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 16:56:19,192 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 16:56:19,194 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:19,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:19,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,194 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:19,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:19,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:19,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:19,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:19,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 16:56:19,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51dd032b[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:19,195 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 16:56:19,195 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,196 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,196 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 16:56:19,196 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 16:56:19,196 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:19,196 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,196 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:19,198 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 16:56:19,198 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:19,198 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,198 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 16:56:19,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:19,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:19,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:19,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:19,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:19,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:19,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:19,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:19,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 16:56:19,200 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 16:56:19,200 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:19,200 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:19,200 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 16:56:19,200 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:19,204 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,204 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,204 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:19,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:19,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:19,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:19,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:19,205 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:19,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:19,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:19,206 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:19,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:19,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:19,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,208 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,208 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:19,208 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:19,208 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:19,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:19,208 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 16:56:19,209 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:19,208 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:19,208 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:19,209 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,209 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,209 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:19,209 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 16:56:19,209 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:19,209 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:19,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:19,309 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:19,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:19,319 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:56:19,325 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:56:19,332 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:56:19,333 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:56:19,420 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 16:56:19,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 16:56:19,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 16:56:19,421 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 16:56:19,421 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:19,421 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 16:56:19,422 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 16:56:19,422 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,422 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:19,423 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:19,423 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:19,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:19,423 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:19,423 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 16:56:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:19,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 16:56:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:19,424 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 16:56:19,424 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 16:56:19,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 16:56:19,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 16:56:19,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 16:56:19,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 16:56:19,425 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:19,425 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. started... 2023-02-13 16:56:19,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 16:56:19,425 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 16:56:19,425 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ce2e1b6d8e70b199cea2ad6e7d3e5909 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:56:19,425 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:19,425 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. started... 2023-02-13 16:56:19,425 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:19,425 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing b21a40fdb4ff985de26173896bbc0187 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:56:19,425 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. started... 2023-02-13 16:56:19,425 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c346712ffdda196120c65a73d77e9601 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 16:56:19,475 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/e6bce08ae4b649dbb56e3ad90719ce56 2023-02-13 16:56:19,499 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6bce08ae4b649dbb56e3ad90719ce56 2023-02-13 16:56:19,870 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/5d07a73655d5408a896cc9d4af384e7f 2023-02-13 16:56:19,893 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d07a73655d5408a896cc9d4af384e7f 2023-02-13 16:56:19,909 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/d476c8de48034275affee9f4effbc8cd 2023-02-13 16:56:19,912 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/b99d5a51f0fd45ff9477d59e0382dd79 2023-02-13 16:56:19,913 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d476c8de48034275affee9f4effbc8cd 2023-02-13 16:56:19,913 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/5d07a73655d5408a896cc9d4af384e7f as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/5d07a73655d5408a896cc9d4af384e7f 2023-02-13 16:56:19,915 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b99d5a51f0fd45ff9477d59e0382dd79 2023-02-13 16:56:19,916 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/e6bce08ae4b649dbb56e3ad90719ce56 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e6bce08ae4b649dbb56e3ad90719ce56 2023-02-13 16:56:19,939 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e6bce08ae4b649dbb56e3ad90719ce56 2023-02-13 16:56:19,939 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d07a73655d5408a896cc9d4af384e7f 2023-02-13 16:56:19,939 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e6bce08ae4b649dbb56e3ad90719ce56, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 16:56:19,939 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/5d07a73655d5408a896cc9d4af384e7f, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 16:56:19,941 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/b99d5a51f0fd45ff9477d59e0382dd79 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/b99d5a51f0fd45ff9477d59e0382dd79 2023-02-13 16:56:19,941 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/d476c8de48034275affee9f4effbc8cd as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/d476c8de48034275affee9f4effbc8cd 2023-02-13 16:56:19,945 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d476c8de48034275affee9f4effbc8cd 2023-02-13 16:56:19,946 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/d476c8de48034275affee9f4effbc8cd, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 16:56:19,946 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ce2e1b6d8e70b199cea2ad6e7d3e5909 in 521ms, sequenceid=23, compaction requested=true 2023-02-13 16:56:19,946 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:19,947 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 16:56:19,948 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b99d5a51f0fd45ff9477d59e0382dd79 2023-02-13 16:56:19,948 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/b99d5a51f0fd45ff9477d59e0382dd79, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 16:56:19,949 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b21a40fdb4ff985de26173896bbc0187 in 524ms, sequenceid=23, compaction requested=true 2023-02-13 16:56:19,949 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:19,949 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:19,949 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 16:56:20,270 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/009ab6e7bc7a4a87959e4d282980afe8 2023-02-13 16:56:20,293 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 009ab6e7bc7a4a87959e4d282980afe8 2023-02-13 16:56:21,104 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/77ded14c63c349d4b929e69b49780d25 2023-02-13 16:56:21,107 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77ded14c63c349d4b929e69b49780d25 2023-02-13 16:56:21,108 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/009ab6e7bc7a4a87959e4d282980afe8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/009ab6e7bc7a4a87959e4d282980afe8 2023-02-13 16:56:21,130 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 009ab6e7bc7a4a87959e4d282980afe8 2023-02-13 16:56:21,130 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/009ab6e7bc7a4a87959e4d282980afe8, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 16:56:21,131 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/77ded14c63c349d4b929e69b49780d25 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/77ded14c63c349d4b929e69b49780d25 2023-02-13 16:56:21,134 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77ded14c63c349d4b929e69b49780d25 2023-02-13 16:56:21,134 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/77ded14c63c349d4b929e69b49780d25, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 16:56:21,135 INFO [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c346712ffdda196120c65a73d77e9601 in 1710ms, sequenceid=23, compaction requested=true 2023-02-13 16:56:21,135 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:21,135 DEBUG [rs(jenkins-hbase4.apache.org,33907,1676307219708)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 16:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 16:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 16:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 16:56:21,137 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,137 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:21,137 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,137 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 16:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:21,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33907,1676307219708' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 16:56:21,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@551e9c9f[Count = 0] remaining members to acquire global barrier 2023-02-13 16:56:21,138 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 16:56:21,138 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,140 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,141 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 16:56:21,141 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,141 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 16:56:21,141 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 16:56:21,141 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33907,1676307219708' in zk 2023-02-13 16:56:21,142 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 16:56:21,142 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,142 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:21,142 DEBUG [member: 'jenkins-hbase4.apache.org,33907,1676307219708' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 16:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33907,1676307219708': 2023-02-13 16:56:21,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33907,1676307219708' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 16:56:21,144 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 16:56:21,144 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 16:56:21,144 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 16:56:21,144 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 16:56:21,144 INFO [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 16:56:21,145 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,145 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 16:56:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 16:56:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,145 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 16:56:21,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:21,146 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 16:56:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,147 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 16:56:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 16:56:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:21,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 16:56:21,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 16:56:21,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 16:56:21,149 DEBUG [(jenkins-hbase4.apache.org,38407,1676307218418)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 16:56:21,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 16:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 16:56:21,149 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 16:56:21,250 DEBUG [Listener at localhost/36833] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 16:56:21,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38407] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 16:56:21,251 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:56:21,251 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 is initiating major compaction (all files) 2023-02-13 16:56:21,251 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:56:21,251 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:56:21,252 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 in testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:21,253 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/bcf8b0e8913f4a31afff7dbc65983844, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/b474ede3450e47d4a0a704f00cafc908, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/509167eb4636427ebca6fc21d740df81, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/1cf93ed1a45c45a9a43c9e5c7cce316d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/5d07a73655d5408a896cc9d4af384e7f] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp, totalSize=27.1 K 2023-02-13 16:56:21,253 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting bcf8b0e8913f4a31afff7dbc65983844, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307372817 2023-02-13 16:56:21,254 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting b474ede3450e47d4a0a704f00cafc908, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307374331 2023-02-13 16:56:21,254 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 509167eb4636427ebca6fc21d740df81, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307375842 2023-02-13 16:56:21,254 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 1cf93ed1a45c45a9a43c9e5c7cce316d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307377765 2023-02-13 16:56:21,255 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 5d07a73655d5408a896cc9d4af384e7f, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:56:21,263 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): ce2e1b6d8e70b199cea2ad6e7d3e5909#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:21,699 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del 2023-02-13 16:56:21,699 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del 2023-02-13 16:56:22,149 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b81c9bc47354250b57b76bcead674e6 2023-02-13 16:56:22,150 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family1/0b81c9bc47354250b57b76bcead674e6 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/0b81c9bc47354250b57b76bcead674e6 2023-02-13 16:56:22,171 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b81c9bc47354250b57b76bcead674e6 2023-02-13 16:56:22,172 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 of ce2e1b6d8e70b199cea2ad6e7d3e5909 into 0b81c9bc47354250b57b76bcead674e6(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:22,172 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:22,173 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:56:22,173 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): ce2e1b6d8e70b199cea2ad6e7d3e5909/family2 is initiating major compaction (all files) 2023-02-13 16:56:22,173 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:56:22,173 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:56:22,173 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of ce2e1b6d8e70b199cea2ad6e7d3e5909/family2 in testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:22,173 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/bceeb6aef4cf496183d5f1f997f3f015, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/fdab1b278e224d7194476a74a6cb9b13, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/310354f2b3704e0fa834f637c227efdc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/3a738f6fda5e46b99d15c233047fff9f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/d476c8de48034275affee9f4effbc8cd] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp, totalSize=25.4 K 2023-02-13 16:56:22,173 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting bceeb6aef4cf496183d5f1f997f3f015, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307372817 2023-02-13 16:56:22,174 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting fdab1b278e224d7194476a74a6cb9b13, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307374331 2023-02-13 16:56:22,174 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 310354f2b3704e0fa834f637c227efdc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307375842 2023-02-13 16:56:22,174 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 3a738f6fda5e46b99d15c233047fff9f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307377765 2023-02-13 16:56:22,175 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting d476c8de48034275affee9f4effbc8cd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:56:22,182 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): ce2e1b6d8e70b199cea2ad6e7d3e5909#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:22,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/bcf8b0e8913f4a31afff7dbc65983844, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/b474ede3450e47d4a0a704f00cafc908, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/509167eb4636427ebca6fc21d740df81, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/1cf93ed1a45c45a9a43c9e5c7cce316d, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/5d07a73655d5408a896cc9d4af384e7f] to archive 2023-02-13 16:56:22,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:22,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/bcf8b0e8913f4a31afff7dbc65983844 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/bcf8b0e8913f4a31afff7dbc65983844 2023-02-13 16:56:22,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/b474ede3450e47d4a0a704f00cafc908 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/b474ede3450e47d4a0a704f00cafc908 2023-02-13 16:56:22,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/509167eb4636427ebca6fc21d740df81 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/509167eb4636427ebca6fc21d740df81 2023-02-13 16:56:22,245 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/1cf93ed1a45c45a9a43c9e5c7cce316d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/1cf93ed1a45c45a9a43c9e5c7cce316d 2023-02-13 16:56:22,246 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/5d07a73655d5408a896cc9d4af384e7f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/5d07a73655d5408a896cc9d4af384e7f 2023-02-13 16:56:22,590 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213ec10ec060c264361a1c46a286e1064b5_del 2023-02-13 16:56:22,591 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213ec10ec060c264361a1c46a286e1064b5_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213ec10ec060c264361a1c46a286e1064b5_del 2023-02-13 16:56:23,402 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44deb5107e9e46b9b3cc5e858c2e32e3 2023-02-13 16:56:23,403 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/.tmp/family2/44deb5107e9e46b9b3cc5e858c2e32e3 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/44deb5107e9e46b9b3cc5e858c2e32e3 2023-02-13 16:56:23,406 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44deb5107e9e46b9b3cc5e858c2e32e3 2023-02-13 16:56:23,407 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ce2e1b6d8e70b199cea2ad6e7d3e5909/family2 of ce2e1b6d8e70b199cea2ad6e7d3e5909 into 44deb5107e9e46b9b3cc5e858c2e32e3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:56:23,407 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:23,407 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:56:23,407 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): b21a40fdb4ff985de26173896bbc0187/family1 is initiating major compaction (all files) 2023-02-13 16:56:23,408 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:56:23,408 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:56:23,409 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of b21a40fdb4ff985de26173896bbc0187/family1 in testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:23,409 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/be0919d2895b4a38b9cbb7687c1a4305, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/007425ccdeeb4f6d8b0509b0531e6b76, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/93376628c3bf4a42ad1676f0ba0bc7b6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/b0d30f156a3644a89b3659be40a01ce0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e6bce08ae4b649dbb56e3ad90719ce56] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp, totalSize=27.1 K 2023-02-13 16:56:23,409 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting be0919d2895b4a38b9cbb7687c1a4305, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307372817 2023-02-13 16:56:23,409 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 007425ccdeeb4f6d8b0509b0531e6b76, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307374331 2023-02-13 16:56:23,410 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 93376628c3bf4a42ad1676f0ba0bc7b6, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307375841 2023-02-13 16:56:23,410 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting b0d30f156a3644a89b3659be40a01ce0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307377764 2023-02-13 16:56:23,410 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting e6bce08ae4b649dbb56e3ad90719ce56, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:56:23,418 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): b21a40fdb4ff985de26173896bbc0187#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:23,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/bceeb6aef4cf496183d5f1f997f3f015, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/fdab1b278e224d7194476a74a6cb9b13, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/310354f2b3704e0fa834f637c227efdc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/3a738f6fda5e46b99d15c233047fff9f, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/d476c8de48034275affee9f4effbc8cd] to archive 2023-02-13 16:56:23,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:23,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/bceeb6aef4cf496183d5f1f997f3f015 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/bceeb6aef4cf496183d5f1f997f3f015 2023-02-13 16:56:23,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/fdab1b278e224d7194476a74a6cb9b13 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/fdab1b278e224d7194476a74a6cb9b13 2023-02-13 16:56:23,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/310354f2b3704e0fa834f637c227efdc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/310354f2b3704e0fa834f637c227efdc 2023-02-13 16:56:23,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/3a738f6fda5e46b99d15c233047fff9f to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/3a738f6fda5e46b99d15c233047fff9f 2023-02-13 16:56:23,448 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/d476c8de48034275affee9f4effbc8cd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family2/d476c8de48034275affee9f4effbc8cd 2023-02-13 16:56:23,455 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del 2023-02-13 16:56:23,455 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del 2023-02-13 16:56:24,306 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e5b8e914ad3d4908b6b3a2e44fb3f1fb 2023-02-13 16:56:24,307 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family1/e5b8e914ad3d4908b6b3a2e44fb3f1fb as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e5b8e914ad3d4908b6b3a2e44fb3f1fb 2023-02-13 16:56:24,328 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e5b8e914ad3d4908b6b3a2e44fb3f1fb 2023-02-13 16:56:24,329 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b21a40fdb4ff985de26173896bbc0187/family1 of b21a40fdb4ff985de26173896bbc0187 into e5b8e914ad3d4908b6b3a2e44fb3f1fb(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:24,329 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:24,329 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:56:24,329 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): b21a40fdb4ff985de26173896bbc0187/family2 is initiating major compaction (all files) 2023-02-13 16:56:24,329 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:56:24,329 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:56:24,329 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of b21a40fdb4ff985de26173896bbc0187/family2 in testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:24,329 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/a85e7522e9764045bc9c92eebe70c420, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2873f41693f94fbb891941141532cddd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/c0d377521ae340a195559b824c24062c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/36831349709943ff8b8c1bc0c35a03b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/b99d5a51f0fd45ff9477d59e0382dd79] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp, totalSize=25.4 K 2023-02-13 16:56:24,330 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting a85e7522e9764045bc9c92eebe70c420, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307372817 2023-02-13 16:56:24,330 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 2873f41693f94fbb891941141532cddd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307374331 2023-02-13 16:56:24,330 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting c0d377521ae340a195559b824c24062c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307375841 2023-02-13 16:56:24,331 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 36831349709943ff8b8c1bc0c35a03b9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307377764 2023-02-13 16:56:24,331 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting b99d5a51f0fd45ff9477d59e0382dd79, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:56:24,338 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): b21a40fdb4ff985de26173896bbc0187#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:24,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/be0919d2895b4a38b9cbb7687c1a4305, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/007425ccdeeb4f6d8b0509b0531e6b76, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/93376628c3bf4a42ad1676f0ba0bc7b6, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/b0d30f156a3644a89b3659be40a01ce0, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e6bce08ae4b649dbb56e3ad90719ce56] to archive 2023-02-13 16:56:24,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:24,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/be0919d2895b4a38b9cbb7687c1a4305 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/be0919d2895b4a38b9cbb7687c1a4305 2023-02-13 16:56:24,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/007425ccdeeb4f6d8b0509b0531e6b76 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/007425ccdeeb4f6d8b0509b0531e6b76 2023-02-13 16:56:24,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/93376628c3bf4a42ad1676f0ba0bc7b6 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/93376628c3bf4a42ad1676f0ba0bc7b6 2023-02-13 16:56:24,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/b0d30f156a3644a89b3659be40a01ce0 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/b0d30f156a3644a89b3659be40a01ce0 2023-02-13 16:56:24,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e6bce08ae4b649dbb56e3ad90719ce56 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/e6bce08ae4b649dbb56e3ad90719ce56 2023-02-13 16:56:24,745 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230213ea998c2b0f9d4cb384e687dac55392c6_del 2023-02-13 16:56:24,746 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213ea998c2b0f9d4cb384e687dac55392c6_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213ea998c2b0f9d4cb384e687dac55392c6_del 2023-02-13 16:56:25,559 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2cd74f820f7b4082803172ff833733a4 2023-02-13 16:56:25,560 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/.tmp/family2/2cd74f820f7b4082803172ff833733a4 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2cd74f820f7b4082803172ff833733a4 2023-02-13 16:56:25,563 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2cd74f820f7b4082803172ff833733a4 2023-02-13 16:56:25,563 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b21a40fdb4ff985de26173896bbc0187/family2 of b21a40fdb4ff985de26173896bbc0187 into 2cd74f820f7b4082803172ff833733a4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:56:25,564 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:25,564 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:56:25,564 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): c346712ffdda196120c65a73d77e9601/family1 is initiating major compaction (all files) 2023-02-13 16:56:25,564 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:56:25,564 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:56:25,565 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of c346712ffdda196120c65a73d77e9601/family1 in testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:25,565 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/7b021715bf4447a2abad724cde654e9b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/6240ec83ac944b07bfe3d2c2a78059b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/d94500fd8c114862bae5e19d286829a3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/2f040cc53d324d4185580771ca2282fc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/009ab6e7bc7a4a87959e4d282980afe8] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp, totalSize=27.1 K 2023-02-13 16:56:25,565 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 7b021715bf4447a2abad724cde654e9b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307372818 2023-02-13 16:56:25,565 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 6240ec83ac944b07bfe3d2c2a78059b7, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307374331 2023-02-13 16:56:25,566 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting d94500fd8c114862bae5e19d286829a3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307375842 2023-02-13 16:56:25,566 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 2f040cc53d324d4185580771ca2282fc, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307377765 2023-02-13 16:56:25,566 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 009ab6e7bc7a4a87959e4d282980afe8, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:56:25,574 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): c346712ffdda196120c65a73d77e9601#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:25,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/a85e7522e9764045bc9c92eebe70c420, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2873f41693f94fbb891941141532cddd, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/c0d377521ae340a195559b824c24062c, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/36831349709943ff8b8c1bc0c35a03b9, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/b99d5a51f0fd45ff9477d59e0382dd79] to archive 2023-02-13 16:56:25,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:25,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/a85e7522e9764045bc9c92eebe70c420 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/a85e7522e9764045bc9c92eebe70c420 2023-02-13 16:56:25,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2873f41693f94fbb891941141532cddd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/2873f41693f94fbb891941141532cddd 2023-02-13 16:56:25,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/c0d377521ae340a195559b824c24062c to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/c0d377521ae340a195559b824c24062c 2023-02-13 16:56:25,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/36831349709943ff8b8c1bc0c35a03b9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/36831349709943ff8b8c1bc0c35a03b9 2023-02-13 16:56:25,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/b99d5a51f0fd45ff9477d59e0382dd79 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family2/b99d5a51f0fd45ff9477d59e0382dd79 2023-02-13 16:56:26,013 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del 2023-02-13 16:56:26,014 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del 2023-02-13 16:56:26,872 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aeac6724ff824f278f1a039efc630fb5 2023-02-13 16:56:26,872 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family1/aeac6724ff824f278f1a039efc630fb5 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/aeac6724ff824f278f1a039efc630fb5 2023-02-13 16:56:26,894 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aeac6724ff824f278f1a039efc630fb5 2023-02-13 16:56:26,895 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c346712ffdda196120c65a73d77e9601/family1 of c346712ffdda196120c65a73d77e9601 into aeac6724ff824f278f1a039efc630fb5(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 16:56:26,895 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:26,895 DEBUG [Listener at localhost/36833] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 16:56:26,895 DEBUG [Listener at localhost/36833] regionserver.HStore(1912): c346712ffdda196120c65a73d77e9601/family2 is initiating major compaction (all files) 2023-02-13 16:56:26,895 INFO [Listener at localhost/36833] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 16:56:26,895 INFO [Listener at localhost/36833] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 16:56:26,895 INFO [Listener at localhost/36833] regionserver.HRegion(2258): Starting compaction of c346712ffdda196120c65a73d77e9601/family2 in testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:26,896 INFO [Listener at localhost/36833] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/3a56f97577df4eaa99d20c418c49d522, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/af4d3f0b04f24e4c94079121bf352644, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/4f9ebc1486e841df996c78202413cacc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/6d1a4c3553bb4e00b0907009b83cb545, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/77ded14c63c349d4b929e69b49780d25] into tmpdir=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp, totalSize=25.4 K 2023-02-13 16:56:26,896 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 3a56f97577df4eaa99d20c418c49d522, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676307372818 2023-02-13 16:56:26,896 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting af4d3f0b04f24e4c94079121bf352644, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676307374331 2023-02-13 16:56:26,897 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 4f9ebc1486e841df996c78202413cacc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676307375842 2023-02-13 16:56:26,897 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 6d1a4c3553bb4e00b0907009b83cb545, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676307377765 2023-02-13 16:56:26,897 DEBUG [Listener at localhost/36833] compactions.Compactor(207): Compacting 77ded14c63c349d4b929e69b49780d25, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 16:56:26,905 INFO [Listener at localhost/36833] throttle.PressureAwareThroughputController(145): c346712ffdda196120c65a73d77e9601#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 16:56:26,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/7b021715bf4447a2abad724cde654e9b, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/6240ec83ac944b07bfe3d2c2a78059b7, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/d94500fd8c114862bae5e19d286829a3, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/2f040cc53d324d4185580771ca2282fc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/009ab6e7bc7a4a87959e4d282980afe8] to archive 2023-02-13 16:56:26,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:26,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/7b021715bf4447a2abad724cde654e9b to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/7b021715bf4447a2abad724cde654e9b 2023-02-13 16:56:26,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/6240ec83ac944b07bfe3d2c2a78059b7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/6240ec83ac944b07bfe3d2c2a78059b7 2023-02-13 16:56:26,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/d94500fd8c114862bae5e19d286829a3 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/d94500fd8c114862bae5e19d286829a3 2023-02-13 16:56:26,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/2f040cc53d324d4185580771ca2282fc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/2f040cc53d324d4185580771ca2282fc 2023-02-13 16:56:26,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/009ab6e7bc7a4a87959e4d282980afe8 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/009ab6e7bc7a4a87959e4d282980afe8 2023-02-13 16:56:27,313 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213daa37238d619428c88a816d42336560f_del 2023-02-13 16:56:27,313 INFO [Listener at localhost/36833] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213daa37238d619428c88a816d42336560f_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213daa37238d619428c88a816d42336560f_del 2023-02-13 16:56:27,724 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 215afae0faf64d86b485f6956ea06882 2023-02-13 16:56:27,725 DEBUG [Listener at localhost/36833] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/.tmp/family2/215afae0faf64d86b485f6956ea06882 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/215afae0faf64d86b485f6956ea06882 2023-02-13 16:56:27,728 INFO [Listener at localhost/36833] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 215afae0faf64d86b485f6956ea06882 2023-02-13 16:56:27,729 INFO [Listener at localhost/36833] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c346712ffdda196120c65a73d77e9601/family2 of c346712ffdda196120c65a73d77e9601 into 215afae0faf64d86b485f6956ea06882(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 16:56:27,729 DEBUG [Listener at localhost/36833] regionserver.HRegion(2288): Compaction status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:27,732 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:56:27,735 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:56:27,738 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:56:27,739 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:56:27,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/3a56f97577df4eaa99d20c418c49d522, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/af4d3f0b04f24e4c94079121bf352644, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/4f9ebc1486e841df996c78202413cacc, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/6d1a4c3553bb4e00b0907009b83cb545, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/77ded14c63c349d4b929e69b49780d25] to archive 2023-02-13 16:56:27,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:27,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/3a56f97577df4eaa99d20c418c49d522 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/3a56f97577df4eaa99d20c418c49d522 2023-02-13 16:56:27,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/af4d3f0b04f24e4c94079121bf352644 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/af4d3f0b04f24e4c94079121bf352644 2023-02-13 16:56:27,744 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38407] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x92E\xA1\xDE\xA7\x15\xA0l}\xA1\xE1\xE3\xD1\x15\xC4\xB6"\x10\xD4\x10\xF2\xA18D\x16\xF9S\xA9H{\x85g\x0F7*\x10{\xC6\x95O0\x9B\x1Cg\xD1L:"8l\x01u2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-13 16:56:27,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/4f9ebc1486e841df996c78202413cacc to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/4f9ebc1486e841df996c78202413cacc 2023-02-13 16:56:27,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/6d1a4c3553bb4e00b0907009b83cb545 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/6d1a4c3553bb4e00b0907009b83cb545 2023-02-13 16:56:27,747 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/77ded14c63c349d4b929e69b49780d25 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family2/77ded14c63c349d4b929e69b49780d25 2023-02-13 16:56:27,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 16:56:27,766 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 16:56:27,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:56:27,767 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676307987767 2023-02-13 16:56:27,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 16:56:28,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-13 16:56:28,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del 2023-02-13 16:56:28,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del 2023-02-13 16:56:28,119 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:56:28,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del 2023-02-13 16:56:28,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:28,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] zookeeper.ReadOnlyZKClient(139): Connect 0x6b977450 to 127.0.0.1:64619 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 16:56:28,139 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1dd74c6c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 16:56:28,139 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:29,096 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302138eb5e958159c40889b43ca0b90ce183d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302138eb5e958159c40889b43ca0b90ce183d 2023-02-13 16:56:29,097 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:29,108 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 16:56:29,111 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44842, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 16:56:29,118 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 16:56:29,121 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 16:56:29,161 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/e78a7eab498c406f9705ae5c6fa3bece first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-13 16:56:29,162 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/e78a7eab498c406f9705ae5c6fa3bece}] 2023-02-13 16:56:29,167 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/e78a7eab498c406f9705ae5c6fa3bece for inclusion in ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 2023-02-13 16:56:29,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-13 16:56:29,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 16:56:29,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: 2023-02-13 16:56:29,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/e78a7eab498c406f9705ae5c6fa3bece to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__quosorjl19of45cqtuiacennsf7tu77gnalp3bg3f82n96qgso81mchmk5e85qs6/family1/e78a7eab498c406f9705ae5c6fa3bece 2023-02-13 16:56:29,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__quosorjl19of45cqtuiacennsf7tu77gnalp3bg3f82n96qgso81mchmk5e85qs6/family1/e78a7eab498c406f9705ae5c6fa3bece as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/796a6bdad10448f2a7779a5e873bf483_SeqId_28_ 2023-02-13 16:56:29,192 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__quosorjl19of45cqtuiacennsf7tu77gnalp3bg3f82n96qgso81mchmk5e85qs6/family1/e78a7eab498c406f9705ae5c6fa3bece into ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/796a6bdad10448f2a7779a5e873bf483_SeqId_28_ - updating store file list. 2023-02-13 16:56:29,215 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/796a6bdad10448f2a7779a5e873bf483_SeqId_28_ into ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 2023-02-13 16:56:29,215 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__quosorjl19of45cqtuiacennsf7tu77gnalp3bg3f82n96qgso81mchmk5e85qs6/family1/e78a7eab498c406f9705ae5c6fa3bece into ce2e1b6d8e70b199cea2ad6e7d3e5909/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/family1/796a6bdad10448f2a7779a5e873bf483_SeqId_28_) 2023-02-13 16:56:29,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__quosorjl19of45cqtuiacennsf7tu77gnalp3bg3f82n96qgso81mchmk5e85qs6/family1/e78a7eab498c406f9705ae5c6fa3bece 2023-02-13 16:56:29,220 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:29,221 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:29,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302130b855384f3ee449b8cb3fd3ff8f2487a to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302130b855384f3ee449b8cb3fd3ff8f2487a 2023-02-13 16:56:29,223 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021321ee2d8f4e4f4df7bd8d1631c21e1d05 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021321ee2d8f4e4f4df7bd8d1631c21e1d05 2023-02-13 16:56:29,224 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302133061081a97cc4283aa60c9436671c4fd to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302133061081a97cc4283aa60c9436671c4fd 2023-02-13 16:56:29,225 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d18d1823b96748b3b2e1e39465d6ee91 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d18d1823b96748b3b2e1e39465d6ee91 2023-02-13 16:56:29,225 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 16:56:29,225 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 16:56:29,225 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:29,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:29,359 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:56:29,787 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213a867eac6f64840fca4a09b49b878db78 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213a867eac6f64840fca4a09b49b878db78 2023-02-13 16:56:29,788 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:29,827 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/a6067445955a46a6a9dc6096e2de76a9 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-13 16:56:29,828 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/a6067445955a46a6a9dc6096e2de76a9}] 2023-02-13 16:56:29,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/a6067445955a46a6a9dc6096e2de76a9 for inclusion in c346712ffdda196120c65a73d77e9601/family1 2023-02-13 16:56:29,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-13 16:56:29,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 16:56:29,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for c346712ffdda196120c65a73d77e9601: 2023-02-13 16:56:29,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/a6067445955a46a6a9dc6096e2de76a9 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__8v37n77u88onrn22fg05shv7enmdns3shbbbjcpgb4vppdpq46cauiglrl7gnr0s/family1/a6067445955a46a6a9dc6096e2de76a9 2023-02-13 16:56:29,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__8v37n77u88onrn22fg05shv7enmdns3shbbbjcpgb4vppdpq46cauiglrl7gnr0s/family1/a6067445955a46a6a9dc6096e2de76a9 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/374b578944fb47a69845ebd39a670861_SeqId_28_ 2023-02-13 16:56:29,853 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__8v37n77u88onrn22fg05shv7enmdns3shbbbjcpgb4vppdpq46cauiglrl7gnr0s/family1/a6067445955a46a6a9dc6096e2de76a9 into c346712ffdda196120c65a73d77e9601/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/374b578944fb47a69845ebd39a670861_SeqId_28_ - updating store file list. 2023-02-13 16:56:29,874 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/374b578944fb47a69845ebd39a670861_SeqId_28_ into c346712ffdda196120c65a73d77e9601/family1 2023-02-13 16:56:29,874 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__8v37n77u88onrn22fg05shv7enmdns3shbbbjcpgb4vppdpq46cauiglrl7gnr0s/family1/a6067445955a46a6a9dc6096e2de76a9 into c346712ffdda196120c65a73d77e9601/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/family1/374b578944fb47a69845ebd39a670861_SeqId_28_) 2023-02-13 16:56:29,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__8v37n77u88onrn22fg05shv7enmdns3shbbbjcpgb4vppdpq46cauiglrl7gnr0s/family1/a6067445955a46a6a9dc6096e2de76a9 2023-02-13 16:56:29,876 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:29,877 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:29,878 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302130b1d5c79956242e195ce977ccc675f57 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302130b1d5c79956242e195ce977ccc675f57 2023-02-13 16:56:29,879 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021328d4edac64b84a2a80496f80ae8a19ee to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021328d4edac64b84a2a80496f80ae8a19ee 2023-02-13 16:56:29,879 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135c07db5363f1414eb85f4e7f8dece714 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135c07db5363f1414eb85f4e7f8dece714 2023-02-13 16:56:29,880 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213742b04eb556a4312b2def7201376e288 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213742b04eb556a4312b2def7201376e288 2023-02-13 16:56:29,880 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 16:56:29,880 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 16:56:29,881 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:29,881 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:30,606 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:56:30,833 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e9b886a5c65c477dbffb3e4756f64e69 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213e9b886a5c65c477dbffb3e4756f64e69 2023-02-13 16:56:30,833 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:30,874 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/5f70a156ea884feba65ba5be792afc63 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-13 16:56:30,875 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187., hostname=jenkins-hbase4.apache.org,33907,1676307219708, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/5f70a156ea884feba65ba5be792afc63}] 2023-02-13 16:56:30,877 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(763): Validating hfile at hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/5f70a156ea884feba65ba5be792afc63 for inclusion in b21a40fdb4ff985de26173896bbc0187/family1 2023-02-13 16:56:30,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-13 16:56:30,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 16:56:30,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegion(2445): Flush status journal for b21a40fdb4ff985de26173896bbc0187: 2023-02-13 16:56:30,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/5f70a156ea884feba65ba5be792afc63 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__to99epoie51ip8q517er1h95reetd27cgo1isljanrtakh01kele4vmit0376oku/family1/5f70a156ea884feba65ba5be792afc63 2023-02-13 16:56:30,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__to99epoie51ip8q517er1h95reetd27cgo1isljanrtakh01kele4vmit0376oku/family1/5f70a156ea884feba65ba5be792afc63 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/3c63c427744f4d46b825984f9fe20a08_SeqId_28_ 2023-02-13 16:56:30,901 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(849): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__to99epoie51ip8q517er1h95reetd27cgo1isljanrtakh01kele4vmit0376oku/family1/5f70a156ea884feba65ba5be792afc63 into b21a40fdb4ff985de26173896bbc0187/family1 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/3c63c427744f4d46b825984f9fe20a08_SeqId_28_ - updating store file list. 2023-02-13 16:56:30,922 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(882): Loaded HFile hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/3c63c427744f4d46b825984f9fe20a08_SeqId_28_ into b21a40fdb4ff985de26173896bbc0187/family1 2023-02-13 16:56:30,922 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.HStore(855): Successfully loaded hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__to99epoie51ip8q517er1h95reetd27cgo1isljanrtakh01kele4vmit0376oku/family1/5f70a156ea884feba65ba5be792afc63 into b21a40fdb4ff985de26173896bbc0187/family1 (new location: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/family1/3c63c427744f4d46b825984f9fe20a08_SeqId_28_) 2023-02-13 16:56:30,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/staging/jenkins__testMajorCompactionFromAdmin__to99epoie51ip8q517er1h95reetd27cgo1isljanrtakh01kele4vmit0376oku/family1/5f70a156ea884feba65ba5be792afc63 2023-02-13 16:56:30,924 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:30,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:30,926 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302133d8d2a342fe443fc88e2e43ac9b596a7 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302133d8d2a342fe443fc88e2e43ac9b596a7 2023-02-13 16:56:30,927 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134250f761cf0142d38739e67db1fbbc4d to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134250f761cf0142d38739e67db1fbbc4d 2023-02-13 16:56:30,928 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213768895e483314916bfce59ef9443c351 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213768895e483314916bfce59ef9443c351 2023-02-13 16:56:30,929 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302138301e4b5242c4755830b055338eb8813 to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302138301e4b5242c4755830b055338eb8813 2023-02-13 16:56:30,929 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 16:56:30,929 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 16:56:30,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:30,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 16:56:30,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 16:56:30,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del] 2023-02-13 16:56:30,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:30,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302130ae8e77a301a499f80e439eaef8e1d23_del 2023-02-13 16:56:30,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 16:56:30,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del] 2023-02-13 16:56:30,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:30,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302137ec4d7ce7a2f4e11ba9a2d4b040ab15d_del 2023-02-13 16:56:30,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 16:56:30,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del] 2023-02-13 16:56:30,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 16:56:30,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del to hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021368e317e9c2a44c3f881bb7e05242a01a_del 2023-02-13 16:56:30,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 16:56:30,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676307222729] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 16:56:30,935 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 16:56:30,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.1710 sec 2023-02-13 16:56:30,952 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:56:30,954 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:56:30,956 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 16:56:30,957 DEBUG [Listener at localhost/36833] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 16:56:30,961 INFO [Listener at localhost/36833] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 16:56:30,998 INFO [Listener at localhost/36833] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 255), OpenFileDescriptor=629 (was 644), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=133 (was 99) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=1320 (was 1026) - AvailableMemoryMB LEAK? - 2023-02-13 16:56:30,998 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7defec53 to 127.0.0.1:64619 2023-02-13 16:56:30,998 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:30,998 INFO [Listener at localhost/36833] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 16:56:30,999 INFO [Listener at localhost/36833] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:56:30,999 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0629018b to 127.0.0.1:64619 2023-02-13 16:56:30,999 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:30,999 DEBUG [Listener at localhost/36833] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 16:56:30,999 DEBUG [Listener at localhost/36833] util.JVMClusterUtil(257): Found active master hash=1142375138, stopped=false 2023-02-13 16:56:30,999 INFO [Listener at localhost/36833] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:56:31,002 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 16:56:31,002 INFO [Listener at localhost/36833] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 16:56:31,002 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:56:31,002 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 16:56:31,002 DEBUG [Listener at localhost/36833] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2412281b to 127.0.0.1:64619 2023-02-13 16:56:31,002 DEBUG [Listener at localhost/36833] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:31,002 INFO [Listener at localhost/36833] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33907,1676307219708' ***** 2023-02-13 16:56:31,002 INFO [Listener at localhost/36833] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 16:56:31,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:56:31,003 INFO [RS:0;jenkins-hbase4:33907] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 16:56:31,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 16:56:31,003 INFO [RS:0;jenkins-hbase4:33907] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 16:56:31,003 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 16:56:31,003 INFO [RS:0;jenkins-hbase4:33907] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 16:56:31,003 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for f4c3c0edca48a44c27c6f10e06a93872 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 039a7376bb8abccd044c10092dd55a94 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 3f0364c8ae730af4bb790813ec085cf7 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for cf586ed2ad20940a6fec159be48a9844 2023-02-13 16:56:31,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f4c3c0edca48a44c27c6f10e06a93872, disabling compactions & flushes 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for f662746060867d63effc2a9ff57f8b73 2023-02-13 16:56:31,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for c0c2fdbf028f696dc83568e799a88cc4 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 812c35f3f9014f1f8af17e3d2a3991cd 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 3c2d6fb96f5457e331a804073866942b 2023-02-13 16:56:31,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for a5218e16a3f58aacc78158f34c1f6d13 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 433223184ab0ee1d5c98c521a6bdfa88 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 59af009f928b299d440fa7c710c3bdbd 2023-02-13 16:56:31,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. after waiting 0 ms 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for ce2e1b6d8e70b199cea2ad6e7d3e5909 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for f38f1642b70af5caecff7a16ee5e02ab 2023-02-13 16:56:31,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 8df3bdbebcbb3c1db6bcb4b3d768f221 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for d9834553e07b975c30d650299cbec8b5 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for b21a40fdb4ff985de26173896bbc0187 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for e794949a1013ec890417b7ad56401941 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for a3c541a8500ba1157e878a28258f5899 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for d345489d398503e06fb04841004d8255 2023-02-13 16:56:31,004 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for c346712ffdda196120c65a73d77e9601 2023-02-13 16:56:31,005 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:31,005 DEBUG [RS:0;jenkins-hbase4:33907] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f077a56 to 127.0.0.1:64619 2023-02-13 16:56:31,005 DEBUG [RS:0;jenkins-hbase4:33907] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:31,005 INFO [RS:0;jenkins-hbase4:33907] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 16:56:31,005 INFO [RS:0;jenkins-hbase4:33907] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 16:56:31,005 INFO [RS:0;jenkins-hbase4:33907] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 16:56:31,005 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 16:56:31,006 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-13 16:56:31,006 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 16:56:31,006 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 16:56:31,006 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 16:56:31,006 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 16:56:31,006 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 16:56:31,006 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-13 16:56:31,006 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1505): Waiting on 039a7376bb8abccd044c10092dd55a94, 1588230740, 3c2d6fb96f5457e331a804073866942b, 3f0364c8ae730af4bb790813ec085cf7, 433223184ab0ee1d5c98c521a6bdfa88, 59af009f928b299d440fa7c710c3bdbd, 812c35f3f9014f1f8af17e3d2a3991cd, 8df3bdbebcbb3c1db6bcb4b3d768f221, a3c541a8500ba1157e878a28258f5899, a5218e16a3f58aacc78158f34c1f6d13, b21a40fdb4ff985de26173896bbc0187, c0c2fdbf028f696dc83568e799a88cc4, c346712ffdda196120c65a73d77e9601, ce2e1b6d8e70b199cea2ad6e7d3e5909, cf586ed2ad20940a6fec159be48a9844, d345489d398503e06fb04841004d8255, d9834553e07b975c30d650299cbec8b5, e794949a1013ec890417b7ad56401941, f38f1642b70af5caecff7a16ee5e02ab, f4c3c0edca48a44c27c6f10e06a93872, f662746060867d63effc2a9ff57f8b73 2023-02-13 16:56:31,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyPolicy/f4c3c0edca48a44c27c6f10e06a93872/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 16:56:31,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:56:31,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f4c3c0edca48a44c27c6f10e06a93872: Waiting for close lock at 1676307391004Running coprocessor pre-close hooks at 1676307391004Disabling compacts and flushes for region at 1676307391004Disabling writes for close at 1676307391004Writing region close event to WAL at 1676307391006 (+2 ms)Running coprocessor post-close hooks at 1676307391009 (+3 ms)Closed at 1676307391009 2023-02-13 16:56:31,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1676307262491.f4c3c0edca48a44c27c6f10e06a93872. 2023-02-13 16:56:31,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 039a7376bb8abccd044c10092dd55a94, disabling compactions & flushes 2023-02-13 16:56:31,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:56:31,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:56:31,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. after waiting 0 ms 2023-02-13 16:56:31,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:56:31,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/039a7376bb8abccd044c10092dd55a94/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 16:56:31,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:56:31,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 039a7376bb8abccd044c10092dd55a94: Waiting for close lock at 1676307391010Running coprocessor pre-close hooks at 1676307391010Disabling compacts and flushes for region at 1676307391010Disabling writes for close at 1676307391010Writing region close event to WAL at 1676307391013 (+3 ms)Running coprocessor post-close hooks at 1676307391016 (+3 ms)Closed at 1676307391016 2023-02-13 16:56:31,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1676307312025.039a7376bb8abccd044c10092dd55a94. 2023-02-13 16:56:31,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3f0364c8ae730af4bb790813ec085cf7, disabling compactions & flushes 2023-02-13 16:56:31,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:56:31,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:56:31,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. after waiting 0 ms 2023-02-13 16:56:31,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:56:31,017 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/.tmp/info/ca9213eec1b141498c7fa8f7cfc77bfe 2023-02-13 16:56:31,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithWeeklyPolicy/3f0364c8ae730af4bb790813ec085cf7/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-13 16:56:31,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:56:31,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3f0364c8ae730af4bb790813ec085cf7: Waiting for close lock at 1676307391017Running coprocessor pre-close hooks at 1676307391017Disabling compacts and flushes for region at 1676307391017Disabling writes for close at 1676307391017Writing region close event to WAL at 1676307391017Running coprocessor post-close hooks at 1676307391020 (+3 ms)Closed at 1676307391020 2023-02-13 16:56:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1676307248044.3f0364c8ae730af4bb790813ec085cf7. 2023-02-13 16:56:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cf586ed2ad20940a6fec159be48a9844, disabling compactions & flushes 2023-02-13 16:56:31,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:56:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:56:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. after waiting 0 ms 2023-02-13 16:56:31,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:56:31,021 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9213eec1b141498c7fa8f7cfc77bfe 2023-02-13 16:56:31,021 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 16:56:31,022 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 16:56:31,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/cf586ed2ad20940a6fec159be48a9844/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 16:56:31,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:56:31,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cf586ed2ad20940a6fec159be48a9844: Waiting for close lock at 1676307391021Running coprocessor pre-close hooks at 1676307391021Disabling compacts and flushes for region at 1676307391021Disabling writes for close at 1676307391021Writing region close event to WAL at 1676307391021Running coprocessor post-close hooks at 1676307391024 (+3 ms)Closed at 1676307391025 (+1 ms) 2023-02-13 16:56:31,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1676307312025.cf586ed2ad20940a6fec159be48a9844. 2023-02-13 16:56:31,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f662746060867d63effc2a9ff57f8b73, disabling compactions & flushes 2023-02-13 16:56:31,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:31,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:31,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. after waiting 0 ms 2023-02-13 16:56:31,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:31,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMinorCompactionWithMonthlyPolicy/f662746060867d63effc2a9ff57f8b73/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-13 16:56:31,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:31,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f662746060867d63effc2a9ff57f8b73: Waiting for close lock at 1676307391025Running coprocessor pre-close hooks at 1676307391025Disabling compacts and flushes for region at 1676307391025Disabling writes for close at 1676307391025Writing region close event to WAL at 1676307391026 (+1 ms)Running coprocessor post-close hooks at 1676307391029 (+3 ms)Closed at 1676307391029 2023-02-13 16:56:31,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1676307358529.f662746060867d63effc2a9ff57f8b73. 2023-02-13 16:56:31,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c0c2fdbf028f696dc83568e799a88cc4, disabling compactions & flushes 2023-02-13 16:56:31,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:56:31,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:56:31,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. after waiting 0 ms 2023-02-13 16:56:31,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:56:31,032 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/c0c2fdbf028f696dc83568e799a88cc4/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 16:56:31,033 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:56:31,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c0c2fdbf028f696dc83568e799a88cc4: Waiting for close lock at 1676307391029Running coprocessor pre-close hooks at 1676307391029Disabling compacts and flushes for region at 1676307391029Disabling writes for close at 1676307391029Writing region close event to WAL at 1676307391030 (+1 ms)Running coprocessor post-close hooks at 1676307391032 (+2 ms)Closed at 1676307391033 (+1 ms) 2023-02-13 16:56:31,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1676307339849.c0c2fdbf028f696dc83568e799a88cc4. 2023-02-13 16:56:31,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 812c35f3f9014f1f8af17e3d2a3991cd, disabling compactions & flushes 2023-02-13 16:56:31,033 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:56:31,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:56:31,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. after waiting 0 ms 2023-02-13 16:56:31,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:56:31,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/812c35f3f9014f1f8af17e3d2a3991cd/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-13 16:56:31,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:56:31,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 812c35f3f9014f1f8af17e3d2a3991cd: Waiting for close lock at 1676307391033Running coprocessor pre-close hooks at 1676307391033Disabling compacts and flushes for region at 1676307391033Disabling writes for close at 1676307391033Writing region close event to WAL at 1676307391034 (+1 ms)Running coprocessor post-close hooks at 1676307391036 (+2 ms)Closed at 1676307391036 2023-02-13 16:56:31,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676307290275.812c35f3f9014f1f8af17e3d2a3991cd. 2023-02-13 16:56:31,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3c2d6fb96f5457e331a804073866942b, disabling compactions & flushes 2023-02-13 16:56:31,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:56:31,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:56:31,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. after waiting 0 ms 2023-02-13 16:56:31,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:56:31,039 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-13 16:56:31,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/3c2d6fb96f5457e331a804073866942b/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-13 16:56:31,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:56:31,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3c2d6fb96f5457e331a804073866942b: Waiting for close lock at 1676307391037Running coprocessor pre-close hooks at 1676307391037Disabling compacts and flushes for region at 1676307391037Disabling writes for close at 1676307391037Writing region close event to WAL at 1676307391037Running coprocessor post-close hooks at 1676307391039 (+2 ms)Closed at 1676307391040 (+1 ms) 2023-02-13 16:56:31,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1676307259232.3c2d6fb96f5457e331a804073866942b. 2023-02-13 16:56:31,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a5218e16a3f58aacc78158f34c1f6d13, disabling compactions & flushes 2023-02-13 16:56:31,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:56:31,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:56:31,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. after waiting 0 ms 2023-02-13 16:56:31,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:56:31,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testCompactionWithHFileLink/a5218e16a3f58aacc78158f34c1f6d13/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 16:56:31,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:56:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a5218e16a3f58aacc78158f34c1f6d13: Waiting for close lock at 1676307391040Running coprocessor pre-close hooks at 1676307391040Disabling compacts and flushes for region at 1676307391040Disabling writes for close at 1676307391040Writing region close event to WAL at 1676307391041 (+1 ms)Running coprocessor post-close hooks at 1676307391043 (+2 ms)Closed at 1676307391043 2023-02-13 16:56:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1676307312025.a5218e16a3f58aacc78158f34c1f6d13. 2023-02-13 16:56:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 433223184ab0ee1d5c98c521a6bdfa88, disabling compactions & flushes 2023-02-13 16:56:31,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:56:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:56:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. after waiting 0 ms 2023-02-13 16:56:31,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:56:31,046 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/433223184ab0ee1d5c98c521a6bdfa88/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 16:56:31,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:56:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 433223184ab0ee1d5c98c521a6bdfa88: Waiting for close lock at 1676307391044Running coprocessor pre-close hooks at 1676307391044Disabling compacts and flushes for region at 1676307391044Disabling writes for close at 1676307391044Writing region close event to WAL at 1676307391045 (+1 ms)Running coprocessor post-close hooks at 1676307391047 (+2 ms)Closed at 1676307391047 2023-02-13 16:56:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1676307339849.433223184ab0ee1d5c98c521a6bdfa88. 2023-02-13 16:56:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 59af009f928b299d440fa7c710c3bdbd, disabling compactions & flushes 2023-02-13 16:56:31,047 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:56:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:56:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. after waiting 0 ms 2023-02-13 16:56:31,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:56:31,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/59af009f928b299d440fa7c710c3bdbd/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-13 16:56:31,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:56:31,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 59af009f928b299d440fa7c710c3bdbd: Waiting for close lock at 1676307391047Running coprocessor pre-close hooks at 1676307391047Disabling compacts and flushes for region at 1676307391047Disabling writes for close at 1676307391047Writing region close event to WAL at 1676307391048 (+1 ms)Running coprocessor post-close hooks at 1676307391049 (+1 ms)Closed at 1676307391050 (+1 ms) 2023-02-13 16:56:31,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1676307259232.59af009f928b299d440fa7c710c3bdbd. 2023-02-13 16:56:31,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ce2e1b6d8e70b199cea2ad6e7d3e5909, disabling compactions & flushes 2023-02-13 16:56:31,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:31,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:31,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. after waiting 0 ms 2023-02-13 16:56:31,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:31,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/ce2e1b6d8e70b199cea2ad6e7d3e5909/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 16:56:31,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:31,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ce2e1b6d8e70b199cea2ad6e7d3e5909: Waiting for close lock at 1676307391050Running coprocessor pre-close hooks at 1676307391050Disabling compacts and flushes for region at 1676307391050Disabling writes for close at 1676307391050Writing region close event to WAL at 1676307391051 (+1 ms)Running coprocessor post-close hooks at 1676307391053 (+2 ms)Closed at 1676307391053 2023-02-13 16:56:31,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1676307371673.ce2e1b6d8e70b199cea2ad6e7d3e5909. 2023-02-13 16:56:31,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f38f1642b70af5caecff7a16ee5e02ab, disabling compactions & flushes 2023-02-13 16:56:31,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:56:31,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:56:31,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. after waiting 0 ms 2023-02-13 16:56:31,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:56:31,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/f38f1642b70af5caecff7a16ee5e02ab/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 16:56:31,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:56:31,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f38f1642b70af5caecff7a16ee5e02ab: Waiting for close lock at 1676307391054Running coprocessor pre-close hooks at 1676307391054Disabling compacts and flushes for region at 1676307391054Disabling writes for close at 1676307391054Writing region close event to WAL at 1676307391054Running coprocessor post-close hooks at 1676307391056 (+2 ms)Closed at 1676307391057 (+1 ms) 2023-02-13 16:56:31,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676307222864.f38f1642b70af5caecff7a16ee5e02ab. 2023-02-13 16:56:31,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8df3bdbebcbb3c1db6bcb4b3d768f221, disabling compactions & flushes 2023-02-13 16:56:31,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:56:31,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:56:31,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. after waiting 0 ms 2023-02-13 16:56:31,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:56:31,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8df3bdbebcbb3c1db6bcb4b3d768f221 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-13 16:56:31,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/.tmp/info/48cd17992a3242dc82d4aacd07282c90 2023-02-13 16:56:31,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/.tmp/info/48cd17992a3242dc82d4aacd07282c90 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/info/48cd17992a3242dc82d4aacd07282c90 2023-02-13 16:56:31,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/info/48cd17992a3242dc82d4aacd07282c90, entries=3, sequenceid=7, filesize=4.8 K 2023-02-13 16:56:31,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 8df3bdbebcbb3c1db6bcb4b3d768f221 in 19ms, sequenceid=7, compaction requested=false 2023-02-13 16:56:31,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/namespace/8df3bdbebcbb3c1db6bcb4b3d768f221/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-13 16:56:31,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:56:31,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8df3bdbebcbb3c1db6bcb4b3d768f221: Waiting for close lock at 1676307391057Running coprocessor pre-close hooks at 1676307391057Disabling compacts and flushes for region at 1676307391057Disabling writes for close at 1676307391058 (+1 ms)Obtaining lock to block concurrent updates at 1676307391058Preparing flush snapshotting stores in 8df3bdbebcbb3c1db6bcb4b3d768f221 at 1676307391058Finished memstore snapshotting hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1676307391058Flushing stores of hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. at 1676307391058Flushing 8df3bdbebcbb3c1db6bcb4b3d768f221/info: creating writer at 1676307391058Flushing 8df3bdbebcbb3c1db6bcb4b3d768f221/info: appending metadata at 1676307391059 (+1 ms)Flushing 8df3bdbebcbb3c1db6bcb4b3d768f221/info: closing flushed file at 1676307391059Flushing 8df3bdbebcbb3c1db6bcb4b3d768f221/info: reopening flushed file at 1676307391073 (+14 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 8df3bdbebcbb3c1db6bcb4b3d768f221 in 19ms, sequenceid=7, compaction requested=false at 1676307391077 (+4 ms)Writing region close event to WAL at 1676307391078 (+1 ms)Running coprocessor post-close hooks at 1676307391080 (+2 ms)Closed at 1676307391080 2023-02-13 16:56:31,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676307222005.8df3bdbebcbb3c1db6bcb4b3d768f221. 2023-02-13 16:56:31,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d9834553e07b975c30d650299cbec8b5, disabling compactions & flushes 2023-02-13 16:56:31,081 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:56:31,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:56:31,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. after waiting 0 ms 2023-02-13 16:56:31,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:56:31,083 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithMonthlyPolicy/d9834553e07b975c30d650299cbec8b5/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-13 16:56:31,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:56:31,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d9834553e07b975c30d650299cbec8b5: Waiting for close lock at 1676307391081Running coprocessor pre-close hooks at 1676307391081Disabling compacts and flushes for region at 1676307391081Disabling writes for close at 1676307391081Writing region close event to WAL at 1676307391081Running coprocessor post-close hooks at 1676307391083 (+2 ms)Closed at 1676307391083 2023-02-13 16:56:31,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1676307237820.d9834553e07b975c30d650299cbec8b5. 2023-02-13 16:56:31,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b21a40fdb4ff985de26173896bbc0187, disabling compactions & flushes 2023-02-13 16:56:31,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:31,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:31,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. after waiting 0 ms 2023-02-13 16:56:31,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/b21a40fdb4ff985de26173896bbc0187/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 16:56:31,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b21a40fdb4ff985de26173896bbc0187: Waiting for close lock at 1676307391084Running coprocessor pre-close hooks at 1676307391084Disabling compacts and flushes for region at 1676307391084Disabling writes for close at 1676307391084Writing region close event to WAL at 1676307391085 (+1 ms)Running coprocessor post-close hooks at 1676307391087 (+2 ms)Closed at 1676307391087 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1676307371673.b21a40fdb4ff985de26173896bbc0187. 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e794949a1013ec890417b7ad56401941, disabling compactions & flushes 2023-02-13 16:56:31,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. after waiting 0 ms 2023-02-13 16:56:31,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:56:31,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testScannerOnBulkLoadRefHFiles/e794949a1013ec890417b7ad56401941/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-13 16:56:31,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:56:31,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e794949a1013ec890417b7ad56401941: Waiting for close lock at 1676307391087Running coprocessor pre-close hooks at 1676307391087Disabling compacts and flushes for region at 1676307391087Disabling writes for close at 1676307391087Writing region close event to WAL at 1676307391088 (+1 ms)Running coprocessor post-close hooks at 1676307391090 (+2 ms)Closed at 1676307391090 2023-02-13 16:56:31,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1676307259232.e794949a1013ec890417b7ad56401941. 2023-02-13 16:56:31,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a3c541a8500ba1157e878a28258f5899, disabling compactions & flushes 2023-02-13 16:56:31,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:56:31,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:56:31,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. after waiting 0 ms 2023-02-13 16:56:31,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:56:31,091 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3c541a8500ba1157e878a28258f5899/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-13 16:56:31,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a3c541a8500ba1157e878a28258f5899: Waiting for close lock at 1676307391090Running coprocessor pre-close hooks at 1676307391090Disabling compacts and flushes for region at 1676307391090Disabling writes for close at 1676307391090Writing region close event to WAL at 1676307391091 (+1 ms)Running coprocessor post-close hooks at 1676307391093 (+2 ms)Closed at 1676307391093 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676307274448.a3c541a8500ba1157e878a28258f5899. 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d345489d398503e06fb04841004d8255, disabling compactions & flushes 2023-02-13 16:56:31,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. after waiting 0 ms 2023-02-13 16:56:31,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:56:31,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/ns/testMinorCompaction/d345489d398503e06fb04841004d8255/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 16:56:31,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:56:31,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d345489d398503e06fb04841004d8255: Waiting for close lock at 1676307391093Running coprocessor pre-close hooks at 1676307391093Disabling compacts and flushes for region at 1676307391093Disabling writes for close at 1676307391093Writing region close event to WAL at 1676307391094 (+1 ms)Running coprocessor post-close hooks at 1676307391097 (+3 ms)Closed at 1676307391097 2023-02-13 16:56:31,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1676307339849.d345489d398503e06fb04841004d8255. 2023-02-13 16:56:31,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c346712ffdda196120c65a73d77e9601, disabling compactions & flushes 2023-02-13 16:56:31,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:31,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:31,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. after waiting 0 ms 2023-02-13 16:56:31,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:31,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/default/testMajorCompactionFromAdmin/c346712ffdda196120c65a73d77e9601/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 16:56:31,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:31,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c346712ffdda196120c65a73d77e9601: Waiting for close lock at 1676307391097Running coprocessor pre-close hooks at 1676307391097Disabling compacts and flushes for region at 1676307391097Disabling writes for close at 1676307391097Writing region close event to WAL at 1676307391098 (+1 ms)Running coprocessor post-close hooks at 1676307391100 (+2 ms)Closed at 1676307391100 2023-02-13 16:56:31,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1676307371673.c346712ffdda196120c65a73d77e9601. 2023-02-13 16:56:31,207 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 16:56:31,407 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 16:56:31,428 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/.tmp/rep_barrier/d008dccc385a44aa96a018a372bca4e0 2023-02-13 16:56:31,431 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d008dccc385a44aa96a018a372bca4e0 2023-02-13 16:56:31,607 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 16:56:31,808 DEBUG [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 16:56:31,837 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/.tmp/table/eec7c479630f4a779523e899141492e8 2023-02-13 16:56:31,841 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eec7c479630f4a779523e899141492e8 2023-02-13 16:56:31,841 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/.tmp/info/ca9213eec1b141498c7fa8f7cfc77bfe as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/info/ca9213eec1b141498c7fa8f7cfc77bfe 2023-02-13 16:56:31,845 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9213eec1b141498c7fa8f7cfc77bfe 2023-02-13 16:56:31,846 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/info/ca9213eec1b141498c7fa8f7cfc77bfe, entries=226, sequenceid=130, filesize=37.9 K 2023-02-13 16:56:31,846 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/.tmp/rep_barrier/d008dccc385a44aa96a018a372bca4e0 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/rep_barrier/d008dccc385a44aa96a018a372bca4e0 2023-02-13 16:56:31,850 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d008dccc385a44aa96a018a372bca4e0 2023-02-13 16:56:31,850 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/rep_barrier/d008dccc385a44aa96a018a372bca4e0, entries=1, sequenceid=130, filesize=5.0 K 2023-02-13 16:56:31,850 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/.tmp/table/eec7c479630f4a779523e899141492e8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/table/eec7c479630f4a779523e899141492e8 2023-02-13 16:56:31,853 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eec7c479630f4a779523e899141492e8 2023-02-13 16:56:31,853 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/table/eec7c479630f4a779523e899141492e8, entries=27, sequenceid=130, filesize=6.6 K 2023-02-13 16:56:31,855 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 849ms, sequenceid=130, compaction requested=false 2023-02-13 16:56:31,870 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 16:56:31,872 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 16:56:31,873 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b977450 to 127.0.0.1:64619 2023-02-13 16:56:31,873 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:31,874 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-13 16:56:31,874 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 16:56:31,874 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 16:56:31,875 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676307391006Running coprocessor pre-close hooks at 1676307391006Disabling compacts and flushes for region at 1676307391006Disabling writes for close at 1676307391006Obtaining lock to block concurrent updates at 1676307391006Preparing flush snapshotting stores in 1588230740 at 1676307391006Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1676307391006Flushing stores of hbase:meta,,1.1588230740 at 1676307391007 (+1 ms)Flushing 1588230740/info: creating writer at 1676307391007Flushing 1588230740/info: appending metadata at 1676307391011 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676307391012 (+1 ms)Flushing 1588230740/rep_barrier: creating writer at 1676307391022 (+10 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676307391023 (+1 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676307391023Flushing 1588230740/table: creating writer at 1676307391431 (+408 ms)Flushing 1588230740/table: appending metadata at 1676307391433 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676307391433Flushing 1588230740/info: reopening flushed file at 1676307391842 (+409 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676307391847 (+5 ms)Flushing 1588230740/table: reopening flushed file at 1676307391851 (+4 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 849ms, sequenceid=130, compaction requested=false at 1676307391855 (+4 ms)Writing region close event to WAL at 1676307391871 (+16 ms)Running coprocessor post-close hooks at 1676307391874 (+3 ms)Closed at 1676307391874 2023-02-13 16:56:31,875 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 16:56:32,008 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33907,1676307219708; all regions closed. 2023-02-13 16:56:32,015 DEBUG [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/oldWALs 2023-02-13 16:56:32,015 INFO [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33907%2C1676307219708.meta:.meta(num 1676307221743) 2023-02-13 16:56:32,019 DEBUG [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/oldWALs 2023-02-13 16:56:32,019 INFO [RS:0;jenkins-hbase4:33907] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33907%2C1676307219708:(num 1676307221184) 2023-02-13 16:56:32,019 DEBUG [RS:0;jenkins-hbase4:33907] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:32,019 INFO [RS:0;jenkins-hbase4:33907] regionserver.LeaseManager(133): Closed leases 2023-02-13 16:56:32,019 INFO [RS:0;jenkins-hbase4:33907] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 16:56:32,019 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 16:56:32,020 INFO [RS:0;jenkins-hbase4:33907] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33907 2023-02-13 16:56:32,025 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33907,1676307219708 2023-02-13 16:56:32,025 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:32,025 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 16:56:32,025 ERROR [Listener at localhost/36833-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a3f71be rejected from java.util.concurrent.ThreadPoolExecutor@1e00f621[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:32,026 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33907,1676307219708] 2023-02-13 16:56:32,027 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33907,1676307219708; numProcessing=1 2023-02-13 16:56:32,029 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33907,1676307219708 already deleted, retry=false 2023-02-13 16:56:32,029 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33907,1676307219708 expired; onlineServers=0 2023-02-13 16:56:32,029 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38407,1676307218418' ***** 2023-02-13 16:56:32,029 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 16:56:32,029 DEBUG [M:0;jenkins-hbase4:38407] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@52a48d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 16:56:32,029 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38407,1676307218418 2023-02-13 16:56:32,029 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38407,1676307218418; all regions closed. 2023-02-13 16:56:32,029 DEBUG [M:0;jenkins-hbase4:38407] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 16:56:32,029 DEBUG [M:0;jenkins-hbase4:38407] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 16:56:32,030 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 16:56:32,030 DEBUG [M:0;jenkins-hbase4:38407] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 16:56:32,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676307220850] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676307220850,5,FailOnTimeoutGroup] 2023-02-13 16:56:32,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676307220852] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676307220852,5,FailOnTimeoutGroup] 2023-02-13 16:56:32,030 INFO [M:0;jenkins-hbase4:38407] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 16:56:32,030 INFO [M:0;jenkins-hbase4:38407] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 16:56:32,030 INFO [M:0;jenkins-hbase4:38407] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 16:56:32,031 DEBUG [M:0;jenkins-hbase4:38407] master.HMaster(1502): Stopping service threads 2023-02-13 16:56:32,031 INFO [M:0;jenkins-hbase4:38407] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 16:56:32,031 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 16:56:32,031 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 16:56:32,031 ERROR [M:0;jenkins-hbase4:38407] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-13 16:56:32,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 16:56:32,032 INFO [M:0;jenkins-hbase4:38407] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 16:56:32,032 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 16:56:32,032 DEBUG [M:0;jenkins-hbase4:38407] zookeeper.ZKUtil(398): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 16:56:32,032 WARN [M:0;jenkins-hbase4:38407] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 16:56:32,032 INFO [M:0;jenkins-hbase4:38407] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 16:56:32,032 INFO [M:0;jenkins-hbase4:38407] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 16:56:32,032 DEBUG [M:0;jenkins-hbase4:38407] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 16:56:32,032 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:32,032 DEBUG [M:0;jenkins-hbase4:38407] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:32,032 DEBUG [M:0;jenkins-hbase4:38407] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 16:56:32,032 DEBUG [M:0;jenkins-hbase4:38407] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:32,033 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.78 KB heapSize=598.63 KB 2023-02-13 16:56:32,127 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:32,127 INFO [RS:0;jenkins-hbase4:33907] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33907,1676307219708; zookeeper connection closed. 2023-02-13 16:56:32,127 ERROR [Listener at localhost/36833-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@b4077a1 rejected from java.util.concurrent.ThreadPoolExecutor@1e00f621[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:32,128 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): regionserver:33907-0x1004e9a266d0001, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:32,128 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6cd0fc16] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6cd0fc16 2023-02-13 16:56:32,128 ERROR [Listener at localhost/36833-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73bb176b rejected from java.util.concurrent.ThreadPoolExecutor@1e00f621[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:32,128 INFO [Listener at localhost/36833] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-13 16:56:32,440 INFO [M:0;jenkins-hbase4:38407] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.78 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/86fa9bb49718416996a0ce4a7d397ff8 2023-02-13 16:56:32,443 INFO [M:0;jenkins-hbase4:38407] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fa9bb49718416996a0ce4a7d397ff8 2023-02-13 16:56:32,444 DEBUG [M:0;jenkins-hbase4:38407] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/86fa9bb49718416996a0ce4a7d397ff8 as hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/86fa9bb49718416996a0ce4a7d397ff8 2023-02-13 16:56:32,446 INFO [M:0;jenkins-hbase4:38407] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fa9bb49718416996a0ce4a7d397ff8 2023-02-13 16:56:32,447 INFO [M:0;jenkins-hbase4:38407] regionserver.HStore(1080): Added hdfs://localhost:45789/user/jenkins/test-data/4269dfbb-ddf3-763b-7e84-c6a4293024c3/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/86fa9bb49718416996a0ce4a7d397ff8, entries=123, sequenceid=1006, filesize=33.3 K 2023-02-13 16:56:32,447 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegion(2947): Finished flush of dataSize ~512.78 KB/525082, heapSize ~598.62 KB/612984, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 414ms, sequenceid=1006, compaction requested=false 2023-02-13 16:56:32,448 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 16:56:32,448 DEBUG [M:0;jenkins-hbase4:38407] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676307392032Disabling compacts and flushes for region at 1676307392032Disabling writes for close at 1676307392032Obtaining lock to block concurrent updates at 1676307392033 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676307392033Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525082, getHeapSize=612984, getOffHeapSize=0, getCellsCount=1161 at 1676307392033Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676307392033Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676307392033Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676307392036 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676307392036Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676307392444 (+408 ms)Finished flush of dataSize ~512.78 KB/525082, heapSize ~598.62 KB/612984, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 414ms, sequenceid=1006, compaction requested=false at 1676307392447 (+3 ms)Writing region close event to WAL at 1676307392448 (+1 ms)Closed at 1676307392448 2023-02-13 16:56:32,451 INFO [M:0;jenkins-hbase4:38407] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 16:56:32,451 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 16:56:32,451 INFO [M:0;jenkins-hbase4:38407] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38407 2023-02-13 16:56:32,453 DEBUG [M:0;jenkins-hbase4:38407] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38407,1676307218418 already deleted, retry=false 2023-02-13 16:56:32,554 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:32,554 INFO [M:0;jenkins-hbase4:38407] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38407,1676307218418; zookeeper connection closed. 2023-02-13 16:56:32,555 ERROR [Listener at localhost/36833-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7c176a1c rejected from java.util.concurrent.ThreadPoolExecutor@62e241f3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:32,555 DEBUG [Listener at localhost/36833-EventThread] zookeeper.ZKWatcher(600): master:38407-0x1004e9a266d0000, quorum=127.0.0.1:64619, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 16:56:32,555 ERROR [Listener at localhost/36833-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4e7b3889 rejected from java.util.concurrent.ThreadPoolExecutor@62e241f3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 16:56:32,556 WARN [Listener at localhost/36833] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 16:56:32,559 INFO [Listener at localhost/36833] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 16:56:32,667 WARN [BP-359752415-172.31.14.131-1676307215066 heartbeating to localhost/127.0.0.1:45789] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 16:56:32,667 WARN [BP-359752415-172.31.14.131-1676307215066 heartbeating to localhost/127.0.0.1:45789] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-359752415-172.31.14.131-1676307215066 (Datanode Uuid 71870c0e-015e-4301-a526-b4af394d64df) service to localhost/127.0.0.1:45789 2023-02-13 16:56:32,670 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/dfs/data/data1/current/BP-359752415-172.31.14.131-1676307215066] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:32,670 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/785db1ea-b41b-c8e7-187d-fce4742f5acd/cluster_6a35e90a-0f12-472d-1eb5-acda82ebdbfc/dfs/data/data2/current/BP-359752415-172.31.14.131-1676307215066] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 16:56:32,696 INFO [Listener at localhost/36833] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 16:56:32,827 INFO [Listener at localhost/36833] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 16:56:32,906 INFO [Listener at localhost/36833] hbase.HBaseTestingUtility(1293): Minicluster is down